2024-03-22T18:41:50.231 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-22T18:41:50.231 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767 -- /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767/orig.config.yaml 2024-03-22T18:41:50.282 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T18:41:50.365 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767 branch: rishabh-yuri-RunCephCmd-base-reef description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7617767' kernel: client: branch: testing flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-yuri-RunCephCmd-base-reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-yuri-RunCephCmd-base-reef sha1: 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6591 sha1: 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 sleep_before_teardown: 0 subset: 10/2000 suite: fs:workload suite_branch: rishabh-yuri-RunCephCmd-base-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1fLBDOwScRL51pal+CDrxMn8om72J7xA68fczNBRNIjdMYBx7xpHj740P2nJtvD3GaIbtx34EX7W4pE6BFqjg= smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6yoapWolvTnexwn4WKYy1CvVsJaJ2G5BKRISTNOlPNqyHDx3R/DuGJ8jL8daoCoDZ2aDqy+iWMk44SLmukBaI= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCRLZXAXZWAzIeSwaF0O7qnhuPPbcHyFWLJCNm1UF85AXrSd79OFiAUOFS+sggOdiqCTbaYO5tnr7xgcNihcyy4= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_17:46:08 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-22T18:41:50.366 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa; will attempt to use it 2024-03-22T18:41:50.366 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks 2024-03-22T18:41:50.366 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-22T18:41:50.368 INFO:teuthology.task.internal:Checking packages... 2024-03-22T18:41:50.390 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46' 2024-03-22T18:41:50.390 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T18:41:50.390 INFO:teuthology.packaging:ref: None 2024-03-22T18:41:50.390 INFO:teuthology.packaging:tag: None 2024-03-22T18:41:50.390 INFO:teuthology.packaging:branch: rishabh-yuri-RunCephCmd-base-reef 2024-03-22T18:41:50.391 INFO:teuthology.packaging:sha1: 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T18:41:50.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-yuri-RunCephCmd-base-reef 2024-03-22T18:41:50.533 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-819.g899aeb57 2024-03-22T18:41:50.533 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-22T18:41:50.540 INFO:teuthology.task.internal:no buildpackages task found 2024-03-22T18:41:50.540 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-22T18:41:50.548 INFO:teuthology.task.internal:Saving configuration 2024-03-22T18:41:50.566 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-22T18:41:50.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T18:41:50.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 18:36:26.918699', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1fLBDOwScRL51pal+CDrxMn8om72J7xA68fczNBRNIjdMYBx7xpHj740P2nJtvD3GaIbtx34EX7W4pE6BFqjg='} 2024-03-22T18:41:50.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 18:36:26.920316', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6yoapWolvTnexwn4WKYy1CvVsJaJ2G5BKRISTNOlPNqyHDx3R/DuGJ8jL8daoCoDZ2aDqy+iWMk44SLmukBaI='} 2024-03-22T18:41:50.643 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 18:36:26.916887', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCRLZXAXZWAzIeSwaF0O7qnhuPPbcHyFWLJCNm1UF85AXrSd79OFiAUOFS+sggOdiqCTbaYO5tnr7xgcNihcyy4='} 2024-03-22T18:41:50.643 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-22T18:41:50.650 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-22T18:41:50.650 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-22T18:41:50.650 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-22T18:41:50.650 INFO:teuthology.run_tasks:Running task console_log... 2024-03-22T18:41:50.752 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb9465e0f70>, signals=[15]) 2024-03-22T18:41:50.753 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-22T18:41:50.759 INFO:teuthology.task.internal:Opening connections... 2024-03-22T18:41:50.759 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T18:41:50.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:41:50.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-03-22T18:41:50.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:41:50.908 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-03-22T18:41:50.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:41:50.977 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-22T18:41:50.985 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-03-22T18:41:51.005 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-03-22T18:41:51.006 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-03-22T18:41:51.065 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2024-03-22T18:41:51.065 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.6 (Ootpa)" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.6" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-22T18:41:51.066 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-22T18:41:51.067 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-22T18:41:51.067 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-03-22T18:41:51.086 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-03-22T18:41:51.106 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-03-22T18:41:51.106 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.6 (Ootpa)" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.6" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T18:41:51.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:41:51.167 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-22T18:41:51.167 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-22T18:41:51.167 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-22T18:41:51.167 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-22T18:41:51.167 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-03-22T18:41:51.185 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-03-22T18:41:51.206 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-03-22T18:41:51.206 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-03-22T18:41:51.265 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2024-03-22T18:41:51.265 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-22T18:41:51.266 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-22T18:41:51.267 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-22T18:41:51.267 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-03-22T18:41:51.285 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-22T18:41:51.298 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-22T18:41:51.305 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-22T18:41:51.305 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-03-22T18:41:51.307 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-03-22T18:41:51.310 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-03-22T18:41:51.328 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-22T18:41:51.335 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-22T18:41:51.335 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-03-22T18:41:51.366 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-03-22T18:41:51.368 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-03-22T18:41:51.395 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-22T18:41:51.471 INFO:teuthology.run_tasks:Running task kernel... 2024-03-22T18:41:51.485 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-22T18:41:51.486 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-22T18:41:51.486 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-22T18:41:51.486 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T18:41:51.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T18:41:51.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-03-22T18:41:51.728 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T18:41:51.728 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-22T18:41:51.728 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-03-22T18:41:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-22T18:41:51.748 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T18:41:51.748 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-22T18:41:51.748 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-03-22T18:41:53.224 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-22T18:41:53.225 INFO:teuthology.orchestra.run.smithi086.stdout:Unable to read consumer identity 2024-03-22T18:41:53.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:41:53.575 INFO:teuthology.orchestra.run.smithi086.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-22T18:41:53.576 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:41:55.230 INFO:teuthology.orchestra.run.smithi086.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-03-22T18:41:55.889 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2024-03-22T18:41:59.523 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 435 kB/s | 24 kB 00:00 2024-03-22T18:42:00.309 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-22T18:42:00.310 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-03-22T18:42:00.310 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-22T18:42:00.455 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-22T18:42:00.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T18:42:00.681 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/8/flavors/default/x86_64/ 2024-03-22T18:42:00.681 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-22T18:42:00.683 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-03-22T18:42:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22 18:42:14 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/8/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [401601496/401601496] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-22T18:42:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-03-22 18:42:14-- 2024-03-22T18:42:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 14s 2024-03-22T18:42:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 383M in 14s (27.6 MB/s) 2024-03-22T18:42:14.873 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-22T18:42:14.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-22T18:42:16.718 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-03-22T18:42:16.809 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-03-22T18:42:16.953 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-03-22T18:42:57.554 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-22T18:42:58.975 INFO:teuthology.orchestra.run.smithi086.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T18:43:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T18:45:33.310 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-22T18:45:33.311 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-22T18:45:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-22T18:45:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-22T18:45:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-22T18:45:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-22T18:45:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-22T18:45:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-22T18:45:33.366 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-22T18:45:33.367 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-22T18:45:33.368 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-22T18:45:33.369 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-22T18:45:33.370 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-22T18:45:33.371 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-22T18:45:33.372 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-22T18:45:33.373 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-22T18:45:33.374 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-22T18:45:33.375 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-22T18:45:33.376 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-22T18:45:33.377 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-22T18:45:33.378 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-22T18:45:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-22T18:45:33.380 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-22T18:45:33.381 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-22T18:45:33.382 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-22T18:45:33.383 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-22T18:45:33.384 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-22T18:45:33.385 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-22T18:45:33.386 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-22T18:45:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-22T18:45:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-22T18:45:33.389 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-22T18:45:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-22T18:45:33.391 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-22T18:45:33.392 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-22T18:45:33.393 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-22T18:45:33.394 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-22T18:45:33.395 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-22T18:45:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-22T18:45:33.397 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-22T18:45:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-22T18:45:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-22T18:45:33.400 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-22T18:45:33.401 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-22T18:45:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-22T18:45:33.403 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-22T18:45:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-22T18:45:33.405 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-22T18:45:33.406 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-22T18:45:33.407 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-22T18:45:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-22T18:45:33.409 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-22T18:45:33.410 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-22T18:45:33.411 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-22T18:45:33.412 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-22T18:45:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-22T18:45:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-22T18:45:33.415 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-22T18:45:33.416 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-22T18:45:33.417 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-22T18:45:33.418 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-22T18:45:33.419 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-22T18:45:33.420 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-22T18:45:33.421 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-22T18:45:33.422 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-22T18:45:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-22T18:45:33.424 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-22T18:45:33.425 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-22T18:45:33.426 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-22T18:45:33.427 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-22T18:45:33.428 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-22T18:45:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-22T18:45:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-22T18:45:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-22T18:45:33.432 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-22T18:45:33.433 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-22T18:45:33.434 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-22T18:45:33.435 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-22T18:45:33.436 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-22T18:45:33.437 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-22T18:45:33.438 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-22T18:45:33.439 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-22T18:45:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-22T18:45:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-22T18:45:33.442 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-22T18:45:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-22T18:45:33.444 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-22T18:45:33.445 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-22T18:45:33.446 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-22T18:45:33.447 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-22T18:45:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-22T18:45:33.449 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-22T18:45:33.450 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-22T18:45:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-22T18:45:33.452 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-22T18:45:33.453 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-22T18:45:33.454 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-22T18:45:33.455 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-22T18:45:33.456 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-22T18:45:33.457 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-22T18:45:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-22T18:45:33.459 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-22T18:45:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-22T18:45:33.461 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-22T18:45:33.462 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-22T18:45:33.463 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-22T18:45:33.464 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-22T18:45:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-22T18:45:33.466 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-22T18:45:33.467 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-22T18:45:33.468 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-22T18:45:33.469 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-22T18:45:33.470 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-22T18:45:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-22T18:45:33.472 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-22T18:45:33.473 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-22T18:45:33.474 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-22T18:45:33.475 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-22T18:45:33.476 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-22T18:45:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-22T18:45:33.478 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-22T18:45:33.479 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-22T18:45:33.480 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-22T18:45:33.481 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-22T18:45:33.482 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-22T18:45:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-22T18:45:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-22T18:45:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-22T18:45:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-22T18:45:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-22T18:45:33.488 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-22T18:45:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-22T18:45:33.490 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-22T18:45:33.491 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-22T18:45:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-22T18:45:33.493 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-22T18:45:33.496 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-22T18:45:33.496 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-03-22T18:45:33.520 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-22T18:45:33.521 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-03-22T18:45:33.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-03-22T18:45:33.595 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-03-22T18:45:33.595 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-03-22T18:45:33.595 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.02 2024-03-22T18:45:33.595 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 123.el8 2024-03-22T18:45:33.595 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Group : System Environment/Base 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 9265450 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Relocations : (not relocatable) 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : Red Hat, Inc. 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : Red Hat, Inc. 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-03-22T18:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:45:33.597 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T18:45:33.598 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T18:45:33.598 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.8.0-g5ec3e6e88b2e 2024-03-22T18:45:33.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T18:45:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-03-22T18:45:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-03-22T18:45:36.143 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-03-22T18:45:36.161 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.sxSVwoQCsR 2024-03-22T18:45:36.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.sxSVwoQCsR 2024-03-22T18:45:36.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.sxSVwoQCsR 2024-03-22T18:45:36.397 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.sxSVwoQCsR is 6KB 2024-03-22T18:45:36.446 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.sxSVwoQCsR 2024-03-22T18:45:36.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T18:45:36.527 INFO:teuthology.orchestra.run.smithi086.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-22T18:45:36.527 INFO:teuthology.orchestra.run.smithi086.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g5ec3e6e88b2e.conf 2024-03-22T18:45:36.527 INFO:teuthology.orchestra.run.smithi086.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-22T18:45:36.527 INFO:teuthology.orchestra.run.smithi086.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-22T18:45:36.527 INFO:teuthology.orchestra.run.smithi086.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-22T18:45:36.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g5ec3e6e88b2e 2024-03-22T18:45:36.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-03-22T18:46:06.690 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-22T18:46:06.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T18:46:06.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:47:06.693 DEBUG:teuthology.orchestra.remote:timed out 2024-03-22T18:47:15.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T18:47:15.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:47:23.328 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-22T18:47:23.837 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T18:47:23.838 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-22T18:47:23.838 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-03-22T18:47:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:6.8.0-g5ec3e6e88b2e 2024-03-22T18:47:23.878 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-22T18:47:23.878 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T18:47:23.878 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-22T18:47:24.880 DEBUG:teuthology.parallel:result is None 2024-03-22T18:47:24.881 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-22T18:47:24.887 INFO:teuthology.task.internal:Creating test directory... 2024-03-22T18:47:24.887 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T18:47:24.891 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T18:47:24.894 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T18:47:24.930 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-22T18:47:24.936 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-22T18:47:24.943 INFO:teuthology.task.internal:Creating archive directory... 2024-03-22T18:47:24.944 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T18:47:24.973 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T18:47:24.976 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T18:47:25.029 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-22T18:47:25.036 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-22T18:47:25.036 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T18:47:25.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T18:47:25.066 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T18:47:25.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T18:47:25.085 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T18:47:25.089 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T18:47:25.109 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T18:47:25.161 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T18:47:25.185 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T18:47:25.200 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T18:47:25.215 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T18:47:25.272 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T18:47:25.303 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T18:47:25.307 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-22T18:47:25.314 INFO:teuthology.task.internal:Configuring sudo... 2024-03-22T18:47:25.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T18:47:25.318 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T18:47:25.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T18:47:25.379 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-22T18:47:25.388 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-22T18:47:25.388 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T18:47:25.392 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T18:47:25.394 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T18:47:25.429 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T18:47:25.539 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T18:47:25.633 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T18:47:25.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T18:47:25.781 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T18:47:25.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T18:47:25.845 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T18:47:25.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T18:47:25.947 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T18:47:26.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T18:47:26.078 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T18:47:26.078 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T18:47:26.147 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T18:47:26.195 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T18:47:26.250 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T18:47:26.335 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T18:47:26.381 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T18:47:26.381 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T18:47:26.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-03-22T18:47:26.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-03-22T18:47:26.456 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-03-22T18:47:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T18:47:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T18:47:26.560 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T18:47:26.984 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-22T18:47:26.993 INFO:teuthology.task.internal:Starting timer... 2024-03-22T18:47:26.993 INFO:teuthology.run_tasks:Running task pcp... 2024-03-22T18:47:27.085 INFO:teuthology.run_tasks:Running task selinux... 2024-03-22T18:47:27.094 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-22T18:47:27.094 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-03-22T18:47:27.206 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-03-22T18:47:27.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-03-22T18:47:27.268 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-03-22T18:47:27.270 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-03-22T18:47:27.336 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-03-22T18:47:27.337 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-22T18:47:27.337 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-03-22T18:47:27.387 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-03-22T18:47:27.387 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-03-22T18:47:27.426 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-03-22T18:47:27.426 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-03-22T18:47:27.450 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-03-22T18:47:27.450 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-03-22T18:47:27.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T18:47:27.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T18:47:27.516 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T18:47:27.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T18:47:27.544 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T18:47:27.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T18:47:27.571 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-22T18:47:27.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-03-22T18:47:27.642 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-03-22T18:47:27.679 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-03-22T18:47:27.714 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-22T18:47:27.724 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-22T18:47:27.730 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-22T18:47:27.849 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-22T18:47:27.862 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-22T18:47:27.864 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi131.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-22T18:58:21.334 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-03-22T18:58:21.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T18:58:21.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:58:21.424 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-22T18:58:21.534 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T18:58:21.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-03-22T18:58:21.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:58:21.609 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-03-22T18:58:21.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-03-22T18:58:21.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T18:58:21.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T18:58:21.706 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-03-22T18:58:21.776 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T18:58:21.776 INFO:teuthology.run_tasks:Running task clock... 2024-03-22T18:58:21.799 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-22T18:58:21.799 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T18:58:21.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T18:58:21.803 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T18:58:21.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T18:58:21.806 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T18:58:21.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T18:58:21.837 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T18:58:21.848 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T18:58:21.853 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T18:58:21.863 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T18:58:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T18:58:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T18:58:21.921 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-03-22T18:58:21.929 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-03-22T18:58:21.933 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-03-22T18:58:21.940 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-03-22T18:58:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T18:58:21.955 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T18:58:21.962 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T18:58:21.969 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T18:58:21.987 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-03-22T18:58:22.014 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-03-22T18:58:22.041 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2024-03-22T18:58:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T18:58:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-03-22T18:58:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.047 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T18:58:22.057 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2024-03-22T18:58:22.061 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T18:58:22.061 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-03-22T18:58:22.062 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.062 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.062 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.062 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.080 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T18:58:22.169 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2024-03-22T18:58:22.176 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T18:58:22.177 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-03-22T18:58:22.177 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.177 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.177 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.177 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T18:58:22.178 INFO:teuthology.run_tasks:Running task pexec... 2024-03-22T18:58:22.188 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-22T18:58:22.188 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T18:58:22.188 DEBUG:teuthology.orchestra.run.smithi086:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-22T18:58:22.189 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2024-03-22T18:58:22.189 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-22T18:58:22.189 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi167.front.sepia.ceph.com 2024-03-22T18:58:22.189 DEBUG:teuthology.orchestra.run.smithi167:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-22T18:58:22.567 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-03-22T18:58:22.568 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2024-03-22T18:58:22.670 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-22T18:58:24.414 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-03-22T18:58:24.546 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-03-22T18:58:24.698 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-03-22T18:58:24.933 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 73 kB/s | 4.5 kB 00:00 2024-03-22T18:58:25.158 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-03-22T18:58:25.484 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-03-22T18:58:27.947 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-03-22T18:58:27.948 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout:Resetting modules: 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-03-22T18:58:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:58:27.980 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout:Resetting modules: 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-03-22T18:58:27.981 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:58:28.392 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout:Resetting modules: 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:58:28.394 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-22T18:58:28.395 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-03-22T18:58:28.395 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:58:36.478 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-22T18:58:36.761 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-03-22T18:58:37.081 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-22T18:58:37.247 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-03-22T18:58:38.962 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-03-22T18:58:39.009 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-03-22T18:58:39.576 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-03-22T18:58:39.731 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-03-22T18:58:40.139 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-03-22T18:58:40.644 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2024-03-22T18:58:42.487 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-03-22T18:58:42.573 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-03-22T18:58:42.595 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-22T18:58:42.733 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-03-22T18:58:42.733 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-03-22T18:58:42.733 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-03-22T18:58:42.733 INFO:teuthology.orchestra.run.smithi131.stdout:Installing group/module packages: 2024-03-22T18:58:42.733 INFO:teuthology.orchestra.run.smithi131.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-22T18:58:42.734 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout:Installing module profiles: 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools/common 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools rhel8 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-03-22T18:58:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:58:42.736 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 26 M 2024-03-22T18:58:42.736 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 87 M 2024-03-22T18:58:42.736 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout:Installing group/module packages: 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-22T18:58:42.765 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-03-22T18:58:42.766 INFO:teuthology.orchestra.run.smithi086.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout:Installing module profiles: 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools/common 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools rhel8 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout:Install 13 Packages 2024-03-22T18:58:42.767 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:58:42.768 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 26 M 2024-03-22T18:58:42.768 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 87 M 2024-03-22T18:58:42.768 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-03-22T18:58:42.865 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-03-22T18:58:42.887 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 259 kB/s | 39 kB 00:00 2024-03-22T18:58:42.903 INFO:teuthology.orchestra.run.smithi086.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 184 kB/s | 25 kB 00:00 2024-03-22T18:58:42.913 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 140 kB/s | 25 kB 00:00 2024-03-22T18:58:42.937 INFO:teuthology.orchestra.run.smithi086.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 558 kB/s | 94 kB 00:00 2024-03-22T18:58:42.938 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 464 kB/s | 94 kB 00:00 2024-03-22T18:58:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 180 kB/s | 39 kB 00:00 2024-03-22T18:58:42.988 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 407 kB/s | 41 kB 00:00 2024-03-22T18:58:43.062 INFO:teuthology.orchestra.run.smithi086.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 258 kB/s | 41 kB 00:00 2024-03-22T18:58:43.063 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 6.7 MB/s | 1.0 MB 00:00 2024-03-22T18:58:43.104 INFO:teuthology.orchestra.run.smithi086.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 6.0 MB/s | 1.0 MB 00:00 2024-03-22T18:58:43.180 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 3.8 MB/s | 739 kB 00:00 2024-03-22T18:58:43.213 INFO:teuthology.orchestra.run.smithi086.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 4.8 MB/s | 739 kB 00:00 2024-03-22T18:58:43.214 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.6 MB/s | 239 kB 00:00 2024-03-22T18:58:43.271 INFO:teuthology.orchestra.run.smithi086.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.4 MB/s | 239 kB 00:00 2024-03-22T18:58:43.356 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 21 MB/s | 8.8 MB 00:00 2024-03-22T18:58:43.373 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 965 kB/s | 154 kB 00:00 2024-03-22T18:58:43.397 INFO:teuthology.orchestra.run.smithi086.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 22 MB/s | 8.8 MB 00:00 2024-03-22T18:58:43.422 INFO:teuthology.orchestra.run.smithi086.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.0 MB/s | 154 kB 00:00 2024-03-22T18:58:43.490 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 12 MB/s | 3.7 MB 00:00 2024-03-22T18:58:43.530 INFO:teuthology.orchestra.run.smithi086.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 12 MB/s | 3.7 MB 00:00 2024-03-22T18:58:43.582 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 12 MB/s | 2.5 MB 00:00 2024-03-22T18:58:43.607 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 424 kB/s | 49 kB 00:00 2024-03-22T18:58:43.664 INFO:teuthology.orchestra.run.smithi086.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 10 MB/s | 2.5 MB 00:00 2024-03-22T18:58:43.722 INFO:teuthology.orchestra.run.smithi086.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 259 kB/s | 49 kB 00:00 2024-03-22T18:58:43.741 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 21 MB/s | 8.1 MB 00:00 2024-03-22T18:58:43.742 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-03-22T18:58:43.742 INFO:teuthology.orchestra.run.smithi131.stdout:Total 25 MB/s | 26 MB 00:01 2024-03-22T18:58:43.839 INFO:teuthology.orchestra.run.smithi086.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 18 MB/s | 8.1 MB 00:00 2024-03-22T18:58:43.840 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-22T18:58:43.840 INFO:teuthology.orchestra.run.smithi086.stdout:Total 24 MB/s | 26 MB 00:01 2024-03-22T18:58:44.030 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-03-22T18:58:44.052 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-03-22T18:58:44.052 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-03-22T18:58:44.283 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-03-22T18:58:44.292 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-03-22T18:58:44.301 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-03-22T18:58:44.325 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-03-22T18:58:44.326 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-03-22T18:58:44.823 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-03-22T18:58:44.832 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-03-22T18:58:44.951 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-03-22T18:58:45.471 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-22T18:58:45.569 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-03-22T18:58:45.651 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-22T18:58:45.836 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-22T18:58:45.866 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-03-22T18:58:45.987 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-22T18:58:46.025 INFO:teuthology.orchestra.run.smithi167.stdout:========================================================================================================== 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout:========================================================================================================== 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout:Installing group/module packages: 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-22T18:58:46.026 INFO:teuthology.orchestra.run.smithi167.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout:Installing module profiles: 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools/common 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools rhel8 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-03-22T18:58:46.027 INFO:teuthology.orchestra.run.smithi167.stdout:========================================================================================================== 2024-03-22T18:58:46.028 INFO:teuthology.orchestra.run.smithi167.stdout:Install 13 Packages 2024-03-22T18:58:46.028 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:58:46.028 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 26 M 2024-03-22T18:58:46.028 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 87 M 2024-03-22T18:58:46.028 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-03-22T18:58:46.095 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-22T18:58:46.140 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-22T18:58:46.213 INFO:teuthology.orchestra.run.smithi167.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 134 kB/s | 25 kB 00:00 2024-03-22T18:58:46.247 INFO:teuthology.orchestra.run.smithi167.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 428 kB/s | 94 kB 00:00 2024-03-22T18:58:46.289 INFO:teuthology.orchestra.run.smithi167.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 151 kB/s | 39 kB 00:00 2024-03-22T18:58:46.292 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-22T18:58:46.306 INFO:teuthology.orchestra.run.smithi167.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 444 kB/s | 41 kB 00:00 2024-03-22T18:58:46.381 INFO:teuthology.orchestra.run.smithi167.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 7.5 MB/s | 1.0 MB 00:00 2024-03-22T18:58:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-22T18:58:46.473 INFO:teuthology.orchestra.run.smithi167.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 4.3 MB/s | 739 kB 00:00 2024-03-22T18:58:46.480 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-22T18:58:46.507 INFO:teuthology.orchestra.run.smithi167.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.9 MB/s | 239 kB 00:00 2024-03-22T18:58:46.619 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-22T18:58:46.649 INFO:teuthology.orchestra.run.smithi167.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 25 MB/s | 8.8 MB 00:00 2024-03-22T18:58:46.655 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-22T18:58:46.674 INFO:teuthology.orchestra.run.smithi167.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 921 kB/s | 154 kB 00:00 2024-03-22T18:58:46.808 INFO:teuthology.orchestra.run.smithi167.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 11 MB/s | 3.7 MB 00:00 2024-03-22T18:58:46.837 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-22T18:58:46.900 INFO:teuthology.orchestra.run.smithi167.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 11 MB/s | 2.5 MB 00:00 2024-03-22T18:58:46.941 INFO:teuthology.orchestra.run.smithi167.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 370 kB/s | 49 kB 00:00 2024-03-22T18:58:47.067 INFO:teuthology.orchestra.run.smithi167.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 20 MB/s | 8.1 MB 00:00 2024-03-22T18:58:47.067 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-03-22T18:58:47.067 INFO:teuthology.orchestra.run.smithi167.stdout:Total 25 MB/s | 26 MB 00:01 2024-03-22T18:58:47.104 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-22T18:58:47.306 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-22T18:58:47.348 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-03-22T18:58:47.370 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-03-22T18:58:47.370 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-03-22T18:58:47.490 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-22T18:58:47.599 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-03-22T18:58:47.609 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-03-22T18:58:47.682 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-22T18:58:48.062 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-22T18:58:48.262 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-22T18:58:48.377 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-03-22T18:58:48.544 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-22T18:58:48.899 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-22T18:58:48.945 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-22T18:58:48.975 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-22T18:58:49.120 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-22T18:58:49.149 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-22T18:58:49.272 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-22T18:58:49.347 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-22T18:58:49.457 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-22T18:58:49.660 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-22T18:58:49.751 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-22T18:58:49.752 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-22T18:58:49.752 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-22T18:58:49.869 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-22T18:58:49.902 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-22T18:58:50.165 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-22T18:58:50.171 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-22T18:58:50.309 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-22T18:58:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-22T18:58:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-22T18:58:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-22T18:58:51.115 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-22T18:58:51.326 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-22T18:58:51.722 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-22T18:58:52.605 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-22T18:58:52.817 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-22T18:58:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-22T18:58:53.441 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-22T18:58:53.441 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-22T18:58:53.441 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-22T18:58:53.441 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-22T18:58:53.442 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-22T18:58:59.588 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-22T18:58:59.589 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:59:00.135 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-03-22T18:59:00.280 DEBUG:teuthology.parallel:result is None 2024-03-22T18:59:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-22T18:59:02.509 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-22T18:59:03.129 INFO:teuthology.orchestra.run.smithi086.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:59:03.130 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-22T18:59:03.378 DEBUG:teuthology.parallel:result is None 2024-03-22T18:59:06.875 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-22T18:59:06.875 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:07.444 INFO:teuthology.orchestra.run.smithi167.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-22T18:59:07.445 INFO:teuthology.orchestra.run.smithi167.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-22T18:59:07.445 INFO:teuthology.orchestra.run.smithi167.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-22T18:59:07.445 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:59:07.445 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-03-22T18:59:07.597 DEBUG:teuthology.parallel:result is None 2024-03-22T18:59:07.598 INFO:teuthology.run_tasks:Running task install... 2024-03-22T18:59:07.612 DEBUG:teuthology.task.install:project ceph 2024-03-22T18:59:07.613 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46'}} 2024-03-22T18:59:07.613 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46'} 2024-03-22T18:59:07.613 INFO:teuthology.task.install:Using flavor: default 2024-03-22T18:59:07.618 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-22T18:59:07.618 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-22T18:59:07.619 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46', 'tag': None, 'wait_for_package': False} 2024-03-22T18:59:07.619 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T18:59:07.621 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46', 'tag': None, 'wait_for_package': False} 2024-03-22T18:59:07.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T18:59:07.622 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46', 'tag': None, 'wait_for_package': False} 2024-03-22T18:59:07.622 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T18:59:07.773 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/ 2024-03-22T18:59:07.773 INFO:teuthology.task.install.rpm:Package version is 18.2.2-819.g899aeb57 2024-03-22T18:59:07.833 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/ 2024-03-22T18:59:07.833 INFO:teuthology.task.install.rpm:Package version is 18.2.2-819.g899aeb57 2024-03-22T18:59:07.847 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/ 2024-03-22T18:59:07.848 INFO:teuthology.task.install.rpm:Package version is 18.2.2-819.g899aeb57 2024-03-22T18:59:07.940 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T18:59:07.940 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T18:59:07.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T18:59:07.944 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T18:59:07.944 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T18:59:07.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T18:59:07.992 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-22T18:59:07.992 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T18:59:08.012 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-22T18:59:08.012 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T18:59:08.015 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T18:59:08.016 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T18:59:08.016 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T18:59:08.044 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-22T18:59:08.044 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T18:59:08.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T18:59:08.112 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T18:59:08.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T18:59:08.144 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T18:59:08.189 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T18:59:08.211 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-03-22T18:59:08.212 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T18:59:08.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-03-22T18:59:08.255 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-03-22T18:59:08.256 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-03-22T18:59:08.276 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-03-22T18:59:08.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-03-22T18:59:08.591 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-03-22T18:59:08.637 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2024-03-22T18:59:08.722 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-22T18:59:10.016 INFO:teuthology.orchestra.run.smithi131.stdout:61 files removed 2024-03-22T18:59:10.016 INFO:teuthology.orchestra.run.smithi167.stdout:61 files removed 2024-03-22T18:59:10.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-22T18:59:10.069 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-22T18:59:10.121 INFO:teuthology.orchestra.run.smithi086.stdout:61 files removed 2024-03-22T18:59:10.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-22T18:59:10.408 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-03-22T18:59:10.409 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2024-03-22T18:59:10.618 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-22T18:59:12.224 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 227 kB/s | 80 kB 00:00 2024-03-22T18:59:12.489 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 144 kB/s | 80 kB 00:00 2024-03-22T18:59:12.738 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 128 kB/s | 80 kB 00:00 2024-03-22T18:59:12.774 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 19 kB/s | 9.9 kB 00:00 2024-03-22T18:59:12.873 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-03-22T18:59:13.055 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 35 kB/s | 9.9 kB 00:00 2024-03-22T18:59:13.108 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 4.8 kB/s | 1.5 kB 00:00 2024-03-22T18:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2024-03-22T18:59:13.300 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-03-22T18:59:13.355 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 5.4 kB/s | 1.5 kB 00:00 2024-03-22T18:59:13.389 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-03-22T18:59:13.515 INFO:teuthology.orchestra.run.smithi086.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-03-22T18:59:14.619 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-03-22T18:59:14.904 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-03-22T18:59:14.977 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-03-22T18:59:18.218 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 453 kB/s | 24 kB 00:00 2024-03-22T18:59:18.476 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 504 kB/s | 24 kB 00:00 2024-03-22T18:59:19.099 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 540 kB/s | 24 kB 00:00 2024-03-22T18:59:19.628 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 66 MB 00:01 2024-03-22T18:59:20.096 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 5.7 MB/s | 8.8 MB 00:01 2024-03-22T18:59:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 59 MB 00:01 2024-03-22T18:59:23.034 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 59 MB 00:01 2024-03-22T18:59:29.604 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 17 MB/s | 8.8 MB 00:00 2024-03-22T18:59:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-22T18:59:32.638 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 59 MB 00:01 2024-03-22T18:59:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 66 MB 00:01 2024-03-22T18:59:35.437 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 MB/s | 66 MB 00:01 2024-03-22T18:59:48.335 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-22T18:59:48.337 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-22T18:59:48.355 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-22T18:59:48.356 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-22T18:59:48.357 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-22T18:59:48.359 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-22T18:59:48.361 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-22T18:59:48.365 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-22T18:59:48.367 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-22T18:59:48.589 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-03-22T18:59:48.603 INFO:teuthology.orchestra.run.smithi167.stdout:=================================================================================================================================== 2024-03-22T18:59:48.603 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout:=================================================================================================================================== 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:18.2.2-819.g899aeb57.el8 ceph 6.4 k 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:18.2.2-819.g899aeb57.el8 ceph 5.4 M 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 852 k 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-819.g899aeb57.el8 ceph 165 k 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:18.2.2-819.g899aeb57.el8 ceph 1.6 M 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 138 k 2024-03-22T18:59:48.604 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 1.7 M 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 7.4 M 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 49 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:18.2.2-819.g899aeb57.el8 ceph 13 M 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:18.2.2-819.g899aeb57.el8 ceph 47 M 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 263 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 172 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:18.2.2-819.g899aeb57.el8 ceph 261 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 18 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:18.2.2-819.g899aeb57.el8 ceph 31 k 2024-03-22T18:59:48.605 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 775 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:18.2.2-819.g899aeb57.el8 ceph 139 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:18.2.2-819.g899aeb57.el8 ceph 197 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:18.2.2-819.g899aeb57.el8 ceph 371 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 366 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:18.2.2-819.g899aeb57.el8 ceph 107 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 95 k 2024-03-22T18:59:48.606 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.5 M 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 181 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.7 M 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.6 M 2024-03-22T18:59:48.607 INFO:teuthology.orchestra.run.smithi167.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:18.2.2-819.g899aeb57.el8 ceph 22 M 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 23 k 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:18.2.2-819.g899aeb57.el8 ceph 2.3 M 2024-03-22T18:59:48.608 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 245 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:18.2.2-819.g899aeb57.el8 ceph 4.6 M 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 18 M 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 15 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:18.2.2-819.g899aeb57.el8 ceph 24 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:18.2.2-819.g899aeb57.el8 ceph 142 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-22T18:59:48.609 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 474 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 6.5 M 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-22T18:59:48.610 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 45 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:18.2.2-819.g899aeb57.el8 ceph 125 k 2024-03-22T18:59:48.611 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-22T18:59:48.612 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-22T18:59:48.613 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-22T18:59:48.614 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql 10 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-03-22T18:59:48.615 INFO:teuthology.orchestra.run.smithi167.stdout:=================================================================================================================================== 2024-03-22T18:59:48.616 INFO:teuthology.orchestra.run.smithi167.stdout:Install 130 Packages 2024-03-22T18:59:48.616 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 16 Packages 2024-03-22T18:59:48.616 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T18:59:48.616 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 212 M 2024-03-22T18:59:48.616 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-03-22T18:59:49.146 INFO:teuthology.orchestra.run.smithi167.stdout:(1/146): ceph-18.2.2-819.g899aeb57.el8.x86_64.r 34 kB/s | 6.4 kB 00:00 2024-03-22T18:59:49.268 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-22T18:59:49.270 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-22T18:59:49.287 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-22T18:59:49.288 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-22T18:59:49.289 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-22T18:59:49.291 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-22T18:59:49.293 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-22T18:59:49.297 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-22T18:59:49.299 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-22T18:59:49.516 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:18.2.2-819.g899aeb57.el8 ceph 6.4 k 2024-03-22T18:59:49.530 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:18.2.2-819.g899aeb57.el8 ceph 5.4 M 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 852 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-819.g899aeb57.el8 ceph 165 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:18.2.2-819.g899aeb57.el8 ceph 1.6 M 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 138 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 1.7 M 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 7.4 M 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 49 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:18.2.2-819.g899aeb57.el8 ceph 13 M 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:18.2.2-819.g899aeb57.el8 ceph 47 M 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 263 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 172 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:18.2.2-819.g899aeb57.el8 ceph 261 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 18 k 2024-03-22T18:59:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:18.2.2-819.g899aeb57.el8 ceph 31 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 775 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:18.2.2-819.g899aeb57.el8 ceph 139 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:18.2.2-819.g899aeb57.el8 ceph 197 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:18.2.2-819.g899aeb57.el8 ceph 371 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 366 k 2024-03-22T18:59:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:18.2.2-819.g899aeb57.el8 ceph 107 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 95 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.5 M 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 181 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.7 M 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.6 M 2024-03-22T18:59:49.533 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:18.2.2-819.g899aeb57.el8 ceph 22 M 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 23 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:18.2.2-819.g899aeb57.el8 ceph 2.3 M 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 245 k 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:18.2.2-819.g899aeb57.el8 ceph 4.6 M 2024-03-22T18:59:49.534 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 18 M 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 15 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:18.2.2-819.g899aeb57.el8 ceph 24 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:18.2.2-819.g899aeb57.el8 ceph 142 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-22T18:59:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 474 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 6.5 M 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-22T18:59:49.536 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 45 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:18.2.2-819.g899aeb57.el8 ceph 125 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-22T18:59:49.537 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-22T18:59:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-22T18:59:49.539 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-22T18:59:49.540 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql 10 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout:Install 130 Packages 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 16 Packages 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 212 M 2024-03-22T18:59:49.541 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-03-22T18:59:49.580 INFO:teuthology.orchestra.run.smithi167.stdout:(2/146): ceph-base-18.2.2-819.g899aeb57.el8.x86 8.7 MB/s | 5.4 MB 00:00 2024-03-22T18:59:49.630 INFO:teuthology.orchestra.run.smithi167.stdout:(3/146): ceph-immutable-object-cache-18.2.2-819 3.2 MB/s | 165 kB 00:00 2024-03-22T18:59:49.781 INFO:teuthology.orchestra.run.smithi167.stdout:(4/146): ceph-fuse-18.2.2-819.g899aeb57.el8.x86 1.3 MB/s | 852 kB 00:00 2024-03-22T18:59:49.856 INFO:teuthology.orchestra.run.smithi167.stdout:(5/146): ceph-mds-18.2.2-819.g899aeb57.el8.x86_ 10 MB/s | 2.3 MB 00:00 2024-03-22T18:59:49.896 INFO:teuthology.orchestra.run.smithi131.stdout:(1/146): ceph-18.2.2-819.g899aeb57.el8.x86_64.r 21 kB/s | 6.4 kB 00:00 2024-03-22T18:59:50.298 INFO:teuthology.orchestra.run.smithi167.stdout:(6/146): ceph-mon-18.2.2-819.g899aeb57.el8.x86_ 10 MB/s | 4.6 MB 00:00 2024-03-22T18:59:50.565 INFO:teuthology.orchestra.run.smithi167.stdout:(7/146): ceph-mgr-18.2.2-819.g899aeb57.el8.x86_ 2.0 MB/s | 1.6 MB 00:00 2024-03-22T18:59:50.755 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-22T18:59:50.756 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-22T18:59:50.775 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-22T18:59:50.777 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-22T18:59:50.779 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-22T18:59:50.780 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-22T18:59:50.783 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-22T18:59:50.787 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-22T18:59:50.789 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-22T18:59:51.018 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-22T18:59:51.032 INFO:teuthology.orchestra.run.smithi167.stdout:(8/146): ceph-common-18.2.2-819.g899aeb57.el8.x 10 MB/s | 22 MB 00:02 2024-03-22T18:59:51.033 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-03-22T18:59:51.033 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-03-22T18:59:51.033 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-03-22T18:59:51.033 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-03-22T18:59:51.033 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-22T18:59:51.033 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:18.2.2-819.g899aeb57.el8 ceph 6.4 k 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:18.2.2-819.g899aeb57.el8 ceph 5.4 M 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 852 k 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-819.g899aeb57.el8 ceph 165 k 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:18.2.2-819.g899aeb57.el8 ceph 1.6 M 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 138 k 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 1.7 M 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 7.4 M 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 49 k 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:18.2.2-819.g899aeb57.el8 ceph 13 M 2024-03-22T18:59:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:18.2.2-819.g899aeb57.el8 ceph 47 M 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 263 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 172 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:18.2.2-819.g899aeb57.el8 ceph 261 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 18 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:18.2.2-819.g899aeb57.el8 ceph 31 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 775 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:18.2.2-819.g899aeb57.el8 ceph 139 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-22T18:59:51.035 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:18.2.2-819.g899aeb57.el8 ceph 197 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:18.2.2-819.g899aeb57.el8 ceph 371 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 366 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:18.2.2-819.g899aeb57.el8 ceph 107 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 95 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.5 M 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 181 k 2024-03-22T18:59:51.036 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.7 M 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 3.6 M 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-22T18:59:51.037 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:18.2.2-819.g899aeb57.el8 ceph 22 M 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 23 k 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:18.2.2-819.g899aeb57.el8 ceph 2.3 M 2024-03-22T18:59:51.038 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 245 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:18.2.2-819.g899aeb57.el8 ceph 4.6 M 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:18.2.2-819.g899aeb57.el8 ceph 18 M 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:18.2.2-819.g899aeb57.el8 ceph-noarch 15 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:18.2.2-819.g899aeb57.el8 ceph 24 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:18.2.2-819.g899aeb57.el8 ceph 142 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-22T18:59:51.039 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 474 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:18.2.2-819.g899aeb57.el8 ceph 6.5 M 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-22T18:59:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-22T18:59:51.041 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:18.2.2-819.g899aeb57.el8 ceph 45 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:18.2.2-819.g899aeb57.el8 ceph 125 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-22T18:59:51.042 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-22T18:59:51.043 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-22T18:59:51.044 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-22T18:59:51.045 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql 10 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout:Install 130 Packages 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 16 Packages 2024-03-22T18:59:51.046 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T18:59:51.047 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 212 M 2024-03-22T18:59:51.047 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-03-22T18:59:51.132 INFO:teuthology.orchestra.run.smithi167.stdout:(9/146): ceph-selinux-18.2.2-819.g899aeb57.el8. 244 kB/s | 24 kB 00:00 2024-03-22T18:59:51.180 INFO:teuthology.orchestra.run.smithi131.stdout:(2/146): ceph-fuse-18.2.2-819.g899aeb57.el8.x86 664 kB/s | 852 kB 00:01 2024-03-22T18:59:51.454 INFO:teuthology.orchestra.run.smithi086.stdout:(1/146): ceph-18.2.2-819.g899aeb57.el8.x86_64.r 19 kB/s | 6.4 kB 00:00 2024-03-22T18:59:51.455 INFO:teuthology.orchestra.run.smithi131.stdout:(3/146): ceph-immutable-object-cache-18.2.2-819 602 kB/s | 165 kB 00:00 2024-03-22T18:59:52.222 INFO:teuthology.orchestra.run.smithi131.stdout:(4/146): ceph-base-18.2.2-819.g899aeb57.el8.x86 2.0 MB/s | 5.4 MB 00:02 2024-03-22T18:59:52.363 INFO:teuthology.orchestra.run.smithi086.stdout:(2/146): ceph-fuse-18.2.2-819.g899aeb57.el8.x86 938 kB/s | 852 kB 00:00 2024-03-22T18:59:52.407 INFO:teuthology.orchestra.run.smithi167.stdout:(10/146): ceph-osd-18.2.2-819.g899aeb57.el8.x86 8.4 MB/s | 18 MB 00:02 2024-03-22T18:59:52.508 INFO:teuthology.orchestra.run.smithi167.stdout:(11/146): cephfs-mirror-18.2.2-819.g899aeb57.el 2.6 MB/s | 261 kB 00:00 2024-03-22T18:59:52.566 INFO:teuthology.orchestra.run.smithi167.stdout:(12/146): libcephfs-devel-18.2.2-819.g899aeb57. 533 kB/s | 31 kB 00:00 2024-03-22T18:59:52.571 INFO:teuthology.orchestra.run.smithi086.stdout:(3/146): ceph-immutable-object-cache-18.2.2-819 796 kB/s | 165 kB 00:00 2024-03-22T18:59:52.878 INFO:teuthology.orchestra.run.smithi167.stdout:(13/146): libcephfs2-18.2.2-819.g899aeb57.el8.x 2.4 MB/s | 775 kB 00:00 2024-03-22T18:59:52.922 INFO:teuthology.orchestra.run.smithi131.stdout:(5/146): ceph-mgr-18.2.2-819.g899aeb57.el8.x86_ 2.2 MB/s | 1.6 MB 00:00 2024-03-22T18:59:52.966 INFO:teuthology.orchestra.run.smithi167.stdout:(14/146): libcephsqlite-18.2.2-819.g899aeb57.el 1.6 MB/s | 142 kB 00:00 2024-03-22T18:59:53.058 INFO:teuthology.orchestra.run.smithi167.stdout:(15/146): librados-devel-18.2.2-819.g899aeb57.e 1.5 MB/s | 139 kB 00:00 2024-03-22T18:59:53.234 INFO:teuthology.orchestra.run.smithi167.stdout:(16/146): libradosstriper1-18.2.2-819.g899aeb57 2.6 MB/s | 474 kB 00:00 2024-03-22T18:59:53.521 INFO:teuthology.orchestra.run.smithi086.stdout:(4/146): ceph-base-18.2.2-819.g899aeb57.el8.x86 2.2 MB/s | 5.4 MB 00:02 2024-03-22T18:59:53.522 INFO:teuthology.orchestra.run.smithi131.stdout:(6/146): ceph-mds-18.2.2-819.g899aeb57.el8.x86_ 1.1 MB/s | 2.3 MB 00:02 2024-03-22T18:59:53.722 INFO:teuthology.orchestra.run.smithi086.stdout:(5/146): ceph-mds-18.2.2-819.g899aeb57.el8.x86_ 2.0 MB/s | 2.3 MB 00:01 2024-03-22T18:59:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:(17/146): ceph-radosgw-18.2.2-819.g899aeb57.el8 3.9 MB/s | 13 MB 00:03 2024-03-22T18:59:53.913 INFO:teuthology.orchestra.run.smithi086.stdout:(6/146): ceph-mgr-18.2.2-819.g899aeb57.el8.x86_ 4.0 MB/s | 1.6 MB 00:00 2024-03-22T18:59:53.976 INFO:teuthology.orchestra.run.smithi167.stdout:(18/146): python3-ceph-argparse-18.2.2-819.g899 489 kB/s | 45 kB 00:00 2024-03-22T18:59:54.093 INFO:teuthology.orchestra.run.smithi167.stdout:(19/146): python3-ceph-common-18.2.2-819.g899ae 1.1 MB/s | 125 kB 00:00 2024-03-22T18:59:54.218 INFO:teuthology.orchestra.run.smithi167.stdout:(20/146): python3-cephfs-18.2.2-819.g899aeb57.e 1.5 MB/s | 197 kB 00:00 2024-03-22T18:59:54.402 INFO:teuthology.orchestra.run.smithi167.stdout:(21/146): python3-rados-18.2.2-819.g899aeb57.el 2.0 MB/s | 371 kB 00:00 2024-03-22T18:59:54.616 INFO:teuthology.orchestra.run.smithi167.stdout:(22/146): python3-rbd-18.2.2-819.g899aeb57.el8. 1.7 MB/s | 366 kB 00:00 2024-03-22T18:59:54.727 INFO:teuthology.orchestra.run.smithi167.stdout:(23/146): python3-rgw-18.2.2-819.g899aeb57.el8. 966 kB/s | 107 kB 00:00 2024-03-22T18:59:54.827 INFO:teuthology.orchestra.run.smithi167.stdout:(24/146): rbd-fuse-18.2.2-819.g899aeb57.el8.x86 950 kB/s | 95 kB 00:00 2024-03-22T18:59:54.938 INFO:teuthology.orchestra.run.smithi086.stdout:(7/146): ceph-mon-18.2.2-819.g899aeb57.el8.x86_ 3.8 MB/s | 4.6 MB 00:01 2024-03-22T18:59:55.114 INFO:teuthology.orchestra.run.smithi131.stdout:(7/146): ceph-mon-18.2.2-819.g899aeb57.el8.x86_ 2.1 MB/s | 4.6 MB 00:02 2024-03-22T18:59:55.128 INFO:teuthology.orchestra.run.smithi167.stdout:(25/146): librgw2-18.2.2-819.g899aeb57.el8.x86_ 3.5 MB/s | 6.5 MB 00:01 2024-03-22T18:59:55.236 INFO:teuthology.orchestra.run.smithi167.stdout:(26/146): rbd-nbd-18.2.2-819.g899aeb57.el8.x86_ 1.6 MB/s | 181 kB 00:00 2024-03-22T18:59:55.328 INFO:teuthology.orchestra.run.smithi167.stdout:(27/146): ceph-grafana-dashboards-18.2.2-819.g8 254 kB/s | 23 kB 00:00 2024-03-22T18:59:55.437 INFO:teuthology.orchestra.run.smithi167.stdout:(28/146): ceph-mgr-cephadm-18.2.2-819.g899aeb57 1.2 MB/s | 138 kB 00:00 2024-03-22T18:59:55.929 INFO:teuthology.orchestra.run.smithi167.stdout:(29/146): rbd-mirror-18.2.2-819.g899aeb57.el8.x 3.1 MB/s | 3.5 MB 00:01 2024-03-22T18:59:56.079 INFO:teuthology.orchestra.run.smithi167.stdout:(30/146): ceph-mgr-dashboard-18.2.2-819.g899aeb 2.7 MB/s | 1.7 MB 00:00 2024-03-22T18:59:56.089 INFO:teuthology.orchestra.run.smithi131.stdout:(8/146): ceph-common-18.2.2-819.g899aeb57.el8.x 3.3 MB/s | 22 MB 00:06 2024-03-22T18:59:56.172 INFO:teuthology.orchestra.run.smithi131.stdout:(9/146): ceph-selinux-18.2.2-819.g899aeb57.el8. 293 kB/s | 24 kB 00:00 2024-03-22T18:59:56.222 INFO:teuthology.orchestra.run.smithi167.stdout:(31/146): ceph-mgr-modules-core-18.2.2-819.g899 1.7 MB/s | 245 kB 00:00 2024-03-22T18:59:56.305 INFO:teuthology.orchestra.run.smithi167.stdout:(32/146): ceph-mgr-rook-18.2.2-819.g899aeb57.el 592 kB/s | 49 kB 00:00 2024-03-22T18:59:56.389 INFO:teuthology.orchestra.run.smithi167.stdout:(33/146): ceph-prometheus-alerts-18.2.2-819.g89 177 kB/s | 15 kB 00:00 2024-03-22T18:59:56.556 INFO:teuthology.orchestra.run.smithi167.stdout:(34/146): ceph-volume-18.2.2-819.g899aeb57.el8. 1.5 MB/s | 263 kB 00:00 2024-03-22T18:59:56.681 INFO:teuthology.orchestra.run.smithi167.stdout:(35/146): cephadm-18.2.2-819.g899aeb57.el8.noar 1.3 MB/s | 172 kB 00:00 2024-03-22T18:59:56.764 INFO:teuthology.orchestra.run.smithi167.stdout:(36/146): cephfs-top-18.2.2-819.g899aeb57.el8.n 216 kB/s | 18 kB 00:00 2024-03-22T18:59:57.081 INFO:teuthology.orchestra.run.smithi086.stdout:(8/146): ceph-radosgw-18.2.2-819.g899aeb57.el8. 6.1 MB/s | 13 MB 00:02 2024-03-22T18:59:57.165 INFO:teuthology.orchestra.run.smithi167.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 765 kB/s | 306 kB 00:00 2024-03-22T18:59:57.172 INFO:teuthology.orchestra.run.smithi086.stdout:(9/146): ceph-selinux-18.2.2-819.g899aeb57.el8. 266 kB/s | 24 kB 00:00 2024-03-22T18:59:57.257 INFO:teuthology.orchestra.run.smithi167.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 643 kB/s | 59 kB 00:00 2024-03-22T18:59:57.340 INFO:teuthology.orchestra.run.smithi167.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2024-03-22T18:59:57.466 INFO:teuthology.orchestra.run.smithi167.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2024-03-22T18:59:57.566 INFO:teuthology.orchestra.run.smithi167.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2024-03-22T18:59:57.641 INFO:teuthology.orchestra.run.smithi167.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2024-03-22T18:59:57.708 INFO:teuthology.orchestra.run.smithi167.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2024-03-22T18:59:57.756 INFO:teuthology.orchestra.run.smithi086.stdout:(10/146): ceph-osd-18.2.2-819.g899aeb57.el8.x86 4.6 MB/s | 18 MB 00:03 2024-03-22T18:59:57.856 INFO:teuthology.orchestra.run.smithi086.stdout:(11/146): cephfs-mirror-18.2.2-819.g899aeb57.el 2.6 MB/s | 261 kB 00:00 2024-03-22T18:59:57.900 INFO:teuthology.orchestra.run.smithi167.stdout:(44/146): ceph-mgr-diskprediction-local-18.2.2- 3.8 MB/s | 7.4 MB 00:01 2024-03-22T18:59:57.948 INFO:teuthology.orchestra.run.smithi086.stdout:(12/146): libcephfs-devel-18.2.2-819.g899aeb57. 337 kB/s | 31 kB 00:00 2024-03-22T18:59:58.165 INFO:teuthology.orchestra.run.smithi086.stdout:(13/146): libcephfs2-18.2.2-819.g899aeb57.el8.x 3.5 MB/s | 775 kB 00:00 2024-03-22T18:59:58.167 INFO:teuthology.orchestra.run.smithi167.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 36 kB/s | 16 kB 00:00 2024-03-22T18:59:58.284 INFO:teuthology.orchestra.run.smithi167.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.4 MB/s | 384 kB 00:00 2024-03-22T18:59:58.292 INFO:teuthology.orchestra.run.smithi086.stdout:(14/146): libcephsqlite-18.2.2-819.g899aeb57.el 1.1 MB/s | 142 kB 00:00 2024-03-22T18:59:58.383 INFO:teuthology.orchestra.run.smithi167.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 825 kB/s | 81 kB 00:00 2024-03-22T18:59:58.390 INFO:teuthology.orchestra.run.smithi086.stdout:(15/146): librados-devel-18.2.2-819.g899aeb57.e 1.4 MB/s | 139 kB 00:00 2024-03-22T18:59:58.476 INFO:teuthology.orchestra.run.smithi167.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 119 kB/s | 11 kB 00:00 2024-03-22T18:59:58.502 INFO:teuthology.orchestra.run.smithi167.stdout:(49/146): python3-cheroot-8.5.2-1.el8.noarch.rp 287 kB/s | 173 kB 00:00 2024-03-22T18:59:58.560 INFO:teuthology.orchestra.run.smithi167.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2024-03-22T18:59:58.574 INFO:teuthology.orchestra.run.smithi086.stdout:(16/146): libradosstriper1-18.2.2-819.g899aeb57 2.5 MB/s | 474 kB 00:00 2024-03-22T18:59:59.061 INFO:teuthology.orchestra.run.smithi167.stdout:(51/146): ceph-test-18.2.2-819.g899aeb57.el8.x8 5.9 MB/s | 47 MB 00:07 2024-03-22T18:59:59.086 INFO:teuthology.orchestra.run.smithi167.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 93 kB/s | 49 kB 00:00 2024-03-22T18:59:59.128 INFO:teuthology.orchestra.run.smithi167.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 1.9 MB/s | 1.2 MB 00:00 2024-03-22T18:59:59.153 INFO:teuthology.orchestra.run.smithi167.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 898 kB/s | 60 kB 00:00 2024-03-22T18:59:59.220 INFO:teuthology.orchestra.run.smithi167.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2024-03-22T18:59:59.246 INFO:teuthology.orchestra.run.smithi167.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-03-22T18:59:59.305 INFO:teuthology.orchestra.run.smithi167.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 398 kB/s | 34 kB 00:00 2024-03-22T18:59:59.338 INFO:teuthology.orchestra.run.smithi167.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2024-03-22T18:59:59.380 INFO:teuthology.orchestra.run.smithi167.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 864 kB/s | 64 kB 00:00 2024-03-22T18:59:59.397 INFO:teuthology.orchestra.run.smithi167.stdout:(60/146): python3-more-itertools-7.2.0-3.el8.no 175 kB/s | 59 kB 00:00 2024-03-22T18:59:59.414 INFO:teuthology.orchestra.run.smithi167.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2024-03-22T18:59:59.472 INFO:teuthology.orchestra.run.smithi167.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 259 kB/s | 24 kB 00:00 2024-03-22T18:59:59.498 INFO:teuthology.orchestra.run.smithi167.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2024-03-22T18:59:59.523 INFO:teuthology.orchestra.run.smithi167.stdout:(64/146): python3-tempora-1.14.1-5.el8.noarch.r 232 kB/s | 29 kB 00:00 2024-03-22T18:59:59.610 INFO:teuthology.orchestra.run.smithi167.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2024-03-22T18:59:59.640 INFO:teuthology.orchestra.run.smithi167.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 523 kB/s | 61 kB 00:00 2024-03-22T18:59:59.665 INFO:teuthology.orchestra.run.smithi167.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2024-03-22T18:59:59.682 INFO:teuthology.orchestra.run.smithi167.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2024-03-22T18:59:59.724 INFO:teuthology.orchestra.run.smithi167.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 274 kB/s | 23 kB 00:00 2024-03-22T18:59:59.750 INFO:teuthology.orchestra.run.smithi167.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00 2024-03-22T18:59:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:(17/146): librgw2-18.2.2-819.g899aeb57.el8.x86_ 5.4 MB/s | 6.5 MB 00:01 2024-03-22T18:59:59.834 INFO:teuthology.orchestra.run.smithi167.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2024-03-22T18:59:59.839 INFO:teuthology.orchestra.run.smithi131.stdout:(10/146): ceph-radosgw-18.2.2-819.g899aeb57.el8 2.8 MB/s | 13 MB 00:04 2024-03-22T18:59:59.850 INFO:teuthology.orchestra.run.smithi086.stdout:(18/146): python3-ceph-argparse-18.2.2-819.g899 670 kB/s | 45 kB 00:00 2024-03-22T18:59:59.898 INFO:teuthology.orchestra.run.smithi167.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 103 kB/s | 18 kB 00:00 2024-03-22T18:59:59.933 INFO:teuthology.orchestra.run.smithi086.stdout:(19/146): python3-ceph-common-18.2.2-819.g899ae 1.5 MB/s | 125 kB 00:00 2024-03-22T18:59:59.934 INFO:teuthology.orchestra.run.smithi167.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 240 kB/s | 43 kB 00:00 2024-03-22T18:59:59.956 INFO:teuthology.orchestra.run.smithi131.stdout:(11/146): cephfs-mirror-18.2.2-819.g899aeb57.el 2.2 MB/s | 261 kB 00:00 2024-03-22T19:00:00.017 INFO:teuthology.orchestra.run.smithi086.stdout:(20/146): python3-cephfs-18.2.2-819.g899aeb57.e 2.3 MB/s | 197 kB 00:00 2024-03-22T19:00:00.059 INFO:teuthology.orchestra.run.smithi167.stdout:(74/146): python3-requests-oauthlib-1.0.0-1.el8 342 kB/s | 43 kB 00:00 2024-03-22T19:00:00.067 INFO:teuthology.orchestra.run.smithi131.stdout:(12/146): libcephfs-devel-18.2.2-819.g899aeb57. 279 kB/s | 31 kB 00:00 2024-03-22T19:00:00.084 INFO:teuthology.orchestra.run.smithi167.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 603 kB/s | 109 kB 00:00 2024-03-22T19:00:00.134 INFO:teuthology.orchestra.run.smithi167.stdout:(76/146): python3-oauthlib-2.1.0-1.el8.noarch.r 515 kB/s | 155 kB 00:00 2024-03-22T19:00:00.202 INFO:teuthology.orchestra.run.smithi167.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 372 kB/s | 44 kB 00:00 2024-03-22T19:00:00.235 INFO:teuthology.orchestra.run.smithi167.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 106 kB/s | 19 kB 00:00 2024-03-22T19:00:00.240 INFO:teuthology.orchestra.run.smithi131.stdout:(13/146): libcephfs2-18.2.2-819.g899aeb57.el8.x 4.4 MB/s | 775 kB 00:00 2024-03-22T19:00:00.259 INFO:teuthology.orchestra.run.smithi086.stdout:(21/146): ceph-common-18.2.2-819.g899aeb57.el8. 2.4 MB/s | 22 MB 00:09 2024-03-22T19:00:00.268 INFO:teuthology.orchestra.run.smithi167.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2024-03-22T19:00:00.301 INFO:teuthology.orchestra.run.smithi086.stdout:(22/146): python3-rados-18.2.2-819.g899aeb57.el 1.3 MB/s | 371 kB 00:00 2024-03-22T19:00:00.315 INFO:teuthology.orchestra.run.smithi131.stdout:(14/146): libcephsqlite-18.2.2-819.g899aeb57.el 1.9 MB/s | 142 kB 00:00 2024-03-22T19:00:00.352 INFO:teuthology.orchestra.run.smithi167.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-03-22T19:00:00.368 INFO:teuthology.orchestra.run.smithi086.stdout:(23/146): python3-rbd-18.2.2-819.g899aeb57.el8. 3.3 MB/s | 366 kB 00:00 2024-03-22T19:00:00.377 INFO:teuthology.orchestra.run.smithi167.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-03-22T19:00:00.393 INFO:teuthology.orchestra.run.smithi086.stdout:(24/146): python3-rgw-18.2.2-819.g899aeb57.el8. 1.1 MB/s | 107 kB 00:00 2024-03-22T19:00:00.403 INFO:teuthology.orchestra.run.smithi167.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2024-03-22T19:00:00.407 INFO:teuthology.orchestra.run.smithi131.stdout:(15/146): librados-devel-18.2.2-819.g899aeb57.e 1.5 MB/s | 139 kB 00:00 2024-03-22T19:00:00.410 INFO:teuthology.orchestra.run.smithi086.stdout:(25/146): rbd-fuse-18.2.2-819.g899aeb57.el8.x86 2.2 MB/s | 95 kB 00:00 2024-03-22T19:00:00.428 INFO:teuthology.orchestra.run.smithi167.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 636 kB/s | 48 kB 00:00 2024-03-22T19:00:00.486 INFO:teuthology.orchestra.run.smithi086.stdout:(26/146): rbd-nbd-18.2.2-819.g899aeb57.el8.x86_ 2.4 MB/s | 181 kB 00:00 2024-03-22T19:00:00.512 INFO:teuthology.orchestra.run.smithi167.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 649 kB/s | 87 kB 00:00 2024-03-22T19:00:00.537 INFO:teuthology.orchestra.run.smithi167.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 430 kB/s | 58 kB 00:00 2024-03-22T19:00:00.580 INFO:teuthology.orchestra.run.smithi086.stdout:(27/146): ceph-grafana-dashboards-18.2.2-819.g8 247 kB/s | 23 kB 00:00 2024-03-22T19:00:00.588 INFO:teuthology.orchestra.run.smithi167.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 243 kB/s | 39 kB 00:00 2024-03-22T19:00:00.599 INFO:teuthology.orchestra.run.smithi131.stdout:(16/146): libradosstriper1-18.2.2-819.g899aeb57 2.4 MB/s | 474 kB 00:00 2024-03-22T19:00:00.653 INFO:teuthology.orchestra.run.smithi086.stdout:(28/146): ceph-mgr-cephadm-18.2.2-819.g899aeb57 1.9 MB/s | 138 kB 00:00 2024-03-22T19:00:00.679 INFO:teuthology.orchestra.run.smithi167.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 496 kB/s | 83 kB 00:00 2024-03-22T19:00:00.713 INFO:teuthology.orchestra.run.smithi167.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2024-03-22T19:00:00.776 INFO:teuthology.orchestra.run.smithi167.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 917 kB/s | 172 kB 00:00 2024-03-22T19:00:00.864 INFO:teuthology.orchestra.run.smithi167.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-22T19:00:00.889 INFO:teuthology.orchestra.run.smithi167.stdout:(91/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 477 kB/s | 100 kB 00:00 2024-03-22T19:00:00.914 INFO:teuthology.orchestra.run.smithi167.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 526 kB/s | 73 kB 00:00 2024-03-22T19:00:00.981 INFO:teuthology.orchestra.run.smithi167.stdout:(93/146): NetworkManager-initscripts-updown-1.4 1.2 MB/s | 144 kB 00:00 2024-03-22T19:00:01.081 INFO:teuthology.orchestra.run.smithi167.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.3 MB/s | 562 kB 00:00 2024-03-22T19:00:01.107 INFO:teuthology.orchestra.run.smithi167.stdout:(95/146): elfutils-libelf-devel-0.189-3.el8.x86 284 kB/s | 62 kB 00:00 2024-03-22T19:00:01.111 INFO:teuthology.orchestra.run.smithi086.stdout:(29/146): rbd-mirror-18.2.2-819.g899aeb57.el8.x 4.8 MB/s | 3.5 MB 00:00 2024-03-22T19:00:01.174 INFO:teuthology.orchestra.run.smithi167.stdout:(96/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-03-22T19:00:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:(30/146): ceph-mgr-dashboard-18.2.2-819.g899aeb 3.3 MB/s | 1.7 MB 00:00 2024-03-22T19:00:01.199 INFO:teuthology.orchestra.run.smithi167.stdout:(97/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 402 kB/s | 47 kB 00:00 2024-03-22T19:00:01.307 INFO:teuthology.orchestra.run.smithi167.stdout:(98/146): python3-cryptography-3.2.1-7.el8_9.x8 2.7 MB/s | 559 kB 00:00 2024-03-22T19:00:01.312 INFO:teuthology.orchestra.run.smithi086.stdout:(31/146): ceph-mgr-modules-core-18.2.2-819.g899 1.9 MB/s | 245 kB 00:00 2024-03-22T19:00:01.366 INFO:teuthology.orchestra.run.smithi167.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2024-03-22T19:00:01.387 INFO:teuthology.orchestra.run.smithi086.stdout:(32/146): ceph-mgr-rook-18.2.2-819.g899aeb57.el 657 kB/s | 49 kB 00:00 2024-03-22T19:00:01.400 INFO:teuthology.orchestra.run.smithi167.stdout:(100/146): python3-packaging-16.8-9.el8.noarch. 213 kB/s | 48 kB 00:00 2024-03-22T19:00:01.442 INFO:teuthology.orchestra.run.smithi167.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 328 kB/s | 44 kB 00:00 2024-03-22T19:00:01.501 INFO:teuthology.orchestra.run.smithi086.stdout:(33/146): ceph-prometheus-alerts-18.2.2-819.g89 129 kB/s | 15 kB 00:00 2024-03-22T19:00:01.525 INFO:teuthology.orchestra.run.smithi167.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 692 kB/s | 110 kB 00:00 2024-03-22T19:00:01.559 INFO:teuthology.orchestra.run.smithi167.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 167 kB/s | 27 kB 00:00 2024-03-22T19:00:01.638 INFO:teuthology.orchestra.run.smithi086.stdout:(34/146): ceph-volume-18.2.2-819.g899aeb57.el8. 1.9 MB/s | 263 kB 00:00 2024-03-22T19:00:01.659 INFO:teuthology.orchestra.run.smithi167.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 582 kB/s | 126 kB 00:00 2024-03-22T19:00:01.721 INFO:teuthology.orchestra.run.smithi086.stdout:(35/146): cephadm-18.2.2-819.g899aeb57.el8.noar 2.0 MB/s | 172 kB 00:00 2024-03-22T19:00:01.768 INFO:teuthology.orchestra.run.smithi167.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 1.8 MB/s | 457 kB 00:00 2024-03-22T19:00:01.797 INFO:teuthology.orchestra.run.smithi086.stdout:(36/146): cephfs-top-18.2.2-819.g899aeb57.el8.n 239 kB/s | 18 kB 00:00 2024-03-22T19:00:01.818 INFO:teuthology.orchestra.run.smithi167.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 2.0 MB/s | 320 kB 00:00 2024-03-22T19:00:01.851 INFO:teuthology.orchestra.run.smithi167.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 2.3 MB/s | 688 kB 00:00 2024-03-22T19:00:01.927 INFO:teuthology.orchestra.run.smithi167.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.3 MB/s | 709 kB 00:00 2024-03-22T19:00:01.985 INFO:teuthology.orchestra.run.smithi167.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2024-03-22T19:00:02.011 INFO:teuthology.orchestra.run.smithi167.stdout:(110/146): python3-markupsafe-0.23-19.el8.x86_6 202 kB/s | 39 kB 00:00 2024-03-22T19:00:02.036 INFO:teuthology.orchestra.run.smithi167.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 590 kB/s | 64 kB 00:00 2024-03-22T19:00:02.070 INFO:teuthology.orchestra.run.smithi167.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.2 MB/s | 103 kB 00:00 2024-03-22T19:00:02.145 INFO:teuthology.orchestra.run.smithi167.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 4.8 MB/s | 538 kB 00:00 2024-03-22T19:00:02.270 INFO:teuthology.orchestra.run.smithi167.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-03-22T19:00:02.304 INFO:teuthology.orchestra.run.smithi167.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2024-03-22T19:00:02.308 INFO:teuthology.orchestra.run.smithi131.stdout:(17/146): ceph-osd-18.2.2-819.g899aeb57.el8.x86 2.0 MB/s | 18 MB 00:08 2024-03-22T19:00:02.371 INFO:teuthology.orchestra.run.smithi167.stdout:(116/146): python3-numpy-1.14.3-10.el8.x86_64.r 12 MB/s | 3.7 MB 00:00 2024-03-22T19:00:02.404 INFO:teuthology.orchestra.run.smithi167.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-03-22T19:00:02.423 INFO:teuthology.orchestra.run.smithi086.stdout:(37/146): ceph-test-18.2.2-819.g899aeb57.el8.x8 8.9 MB/s | 47 MB 00:05 2024-03-22T19:00:02.425 INFO:teuthology.orchestra.run.smithi131.stdout:(18/146): librgw2-18.2.2-819.g899aeb57.el8.x86_ 3.6 MB/s | 6.5 MB 00:01 2024-03-22T19:00:02.450 INFO:teuthology.orchestra.run.smithi131.stdout:(19/146): python3-ceph-argparse-18.2.2-819.g899 315 kB/s | 45 kB 00:00 2024-03-22T19:00:02.463 INFO:teuthology.orchestra.run.smithi167.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2024-03-22T19:00:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:(20/146): python3-ceph-common-18.2.2-819.g899ae 1.8 MB/s | 125 kB 00:00 2024-03-22T19:00:02.517 INFO:teuthology.orchestra.run.smithi131.stdout:(21/146): python3-cephfs-18.2.2-819.g899aeb57.e 2.9 MB/s | 197 kB 00:00 2024-03-22T19:00:02.538 INFO:teuthology.orchestra.run.smithi167.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-03-22T19:00:02.572 INFO:teuthology.orchestra.run.smithi086.stdout:(38/146): ceph-mgr-diskprediction-local-18.2.2- 5.1 MB/s | 7.4 MB 00:01 2024-03-22T19:00:02.592 INFO:teuthology.orchestra.run.smithi131.stdout:(22/146): python3-rados-18.2.2-819.g899aeb57.el 3.6 MB/s | 371 kB 00:00 2024-03-22T19:00:02.622 INFO:teuthology.orchestra.run.smithi167.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 21 MB/s | 5.2 MB 00:00 2024-03-22T19:00:02.626 INFO:teuthology.orchestra.run.smithi131.stdout:(23/146): python3-rbd-18.2.2-819.g899aeb57.el8. 3.3 MB/s | 366 kB 00:00 2024-03-22T19:00:02.659 INFO:teuthology.orchestra.run.smithi131.stdout:(24/146): python3-rgw-18.2.2-819.g899aeb57.el8. 1.6 MB/s | 107 kB 00:00 2024-03-22T19:00:02.685 INFO:teuthology.orchestra.run.smithi131.stdout:(25/146): rbd-fuse-18.2.2-819.g899aeb57.el8.x86 1.6 MB/s | 95 kB 00:00 2024-03-22T19:00:02.764 INFO:teuthology.orchestra.run.smithi167.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-03-22T19:00:02.783 INFO:teuthology.orchestra.run.smithi131.stdout:(26/146): rbd-nbd-18.2.2-819.g899aeb57.el8.x86_ 1.8 MB/s | 181 kB 00:00 2024-03-22T19:00:02.839 INFO:teuthology.orchestra.run.smithi167.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 2.3 MB/s | 714 kB 00:00 2024-03-22T19:00:02.852 INFO:teuthology.orchestra.run.smithi131.stdout:(27/146): ceph-grafana-dashboards-18.2.2-819.g8 338 kB/s | 23 kB 00:00 2024-03-22T19:00:02.865 INFO:teuthology.orchestra.run.smithi167.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 817 kB/s | 198 kB 00:00 2024-03-22T19:00:02.919 INFO:teuthology.orchestra.run.smithi131.stdout:(28/146): ceph-mgr-cephadm-18.2.2-819.g899aeb57 2.0 MB/s | 138 kB 00:00 2024-03-22T19:00:02.940 INFO:teuthology.orchestra.run.smithi167.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-03-22T19:00:03.032 INFO:teuthology.orchestra.run.smithi167.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 282 kB/s | 54 kB 00:00 2024-03-22T19:00:03.073 INFO:teuthology.orchestra.run.smithi086.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 91 kB/s | 59 kB 00:00 2024-03-22T19:00:03.098 INFO:teuthology.orchestra.run.smithi086.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 235 kB/s | 306 kB 00:01 2024-03-22T19:00:03.115 INFO:teuthology.orchestra.run.smithi086.stdout:(41/146): libunwind-1.3.1-3.el8.x86_64.rpm 138 kB/s | 75 kB 00:00 2024-03-22T19:00:03.207 INFO:teuthology.orchestra.run.smithi167.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 705 kB/s | 188 kB 00:00 2024-03-22T19:00:03.249 INFO:teuthology.orchestra.run.smithi167.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 6.9 MB/s | 1.5 MB 00:00 2024-03-22T19:00:03.265 INFO:teuthology.orchestra.run.smithi086.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 261 kB/s | 44 kB 00:00 2024-03-22T19:00:03.319 INFO:teuthology.orchestra.run.smithi131.stdout:(29/146): rbd-mirror-18.2.2-819.g899aeb57.el8.x 5.2 MB/s | 3.5 MB 00:00 2024-03-22T19:00:03.361 INFO:teuthology.orchestra.run.smithi131.stdout:(30/146): ceph-mgr-dashboard-18.2.2-819.g899aeb 3.9 MB/s | 1.7 MB 00:00 2024-03-22T19:00:03.386 INFO:teuthology.orchestra.run.smithi086.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 273 kB/s | 33 kB 00:00 2024-03-22T19:00:03.424 INFO:teuthology.orchestra.run.smithi086.stdout:(44/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2024-03-22T19:00:03.450 INFO:teuthology.orchestra.run.smithi086.stdout:(45/146): python3-beautifulsoup4-4.6.3-2.el8.1. 554 kB/s | 185 kB 00:00 2024-03-22T19:00:03.478 INFO:teuthology.orchestra.run.smithi131.stdout:(31/146): ceph-mgr-modules-core-18.2.2-819.g899 2.1 MB/s | 245 kB 00:00 2024-03-22T19:00:03.483 INFO:teuthology.orchestra.run.smithi086.stdout:(46/146): python3-certifi-2018.10.15-7.el8.noar 169 kB/s | 16 kB 00:00 2024-03-22T19:00:03.516 INFO:teuthology.orchestra.run.smithi167.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 21 MB/s | 14 MB 00:00 2024-03-22T19:00:03.525 INFO:teuthology.orchestra.run.smithi086.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2024-03-22T19:00:03.562 INFO:teuthology.orchestra.run.smithi131.stdout:(32/146): ceph-mgr-rook-18.2.2-819.g899aeb57.el 591 kB/s | 49 kB 00:00 2024-03-22T19:00:03.618 INFO:teuthology.orchestra.run.smithi086.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2024-03-22T19:00:03.628 INFO:teuthology.orchestra.run.smithi131.stdout:(33/146): ceph-prometheus-alerts-18.2.2-819.g89 222 kB/s | 15 kB 00:00 2024-03-22T19:00:03.651 INFO:teuthology.orchestra.run.smithi086.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2024-03-22T19:00:03.676 INFO:teuthology.orchestra.run.smithi086.stdout:(50/146): python3-google-auth-1.1.1-10.el8.noar 423 kB/s | 81 kB 00:00 2024-03-22T19:00:03.710 INFO:teuthology.orchestra.run.smithi086.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2024-03-22T19:00:03.713 INFO:teuthology.orchestra.run.smithi167.stdout:(129/146): postgresql-contrib-10.23-4.module+el 1.6 MB/s | 812 kB 00:00 2024-03-22T19:00:03.737 INFO:teuthology.orchestra.run.smithi131.stdout:(34/146): ceph-volume-18.2.2-819.g899aeb57.el8. 2.4 MB/s | 263 kB 00:00 2024-03-22T19:00:03.777 INFO:teuthology.orchestra.run.smithi086.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 486 kB/s | 49 kB 00:00 2024-03-22T19:00:03.802 INFO:teuthology.orchestra.run.smithi086.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 639 kB/s | 59 kB 00:00 2024-03-22T19:00:03.813 INFO:teuthology.orchestra.run.smithi131.stdout:(35/146): cephadm-18.2.2-819.g899aeb57.el8.noar 2.2 MB/s | 172 kB 00:00 2024-03-22T19:00:03.833 INFO:teuthology.orchestra.run.smithi167.stdout:(130/146): postgresql-server-10.23-4.module+el8 8.7 MB/s | 5.1 MB 00:00 2024-03-22T19:00:03.880 INFO:teuthology.orchestra.run.smithi131.stdout:(36/146): cephfs-top-18.2.2-819.g899aeb57.el8.n 270 kB/s | 18 kB 00:00 2024-03-22T19:00:03.885 INFO:teuthology.orchestra.run.smithi086.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2024-03-22T19:00:03.911 INFO:teuthology.orchestra.run.smithi086.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-03-22T19:00:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2024-03-22T19:00:04.011 INFO:teuthology.orchestra.run.smithi086.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2024-03-22T19:00:04.018 INFO:teuthology.orchestra.run.smithi167.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-03-22T19:00:04.103 INFO:teuthology.orchestra.run.smithi086.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2024-03-22T19:00:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 548 kB/s | 64 kB 00:00 2024-03-22T19:00:04.146 INFO:teuthology.orchestra.run.smithi131.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2024-03-22T19:00:04.175 INFO:teuthology.orchestra.run.smithi167.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-03-22T19:00:04.204 INFO:teuthology.orchestra.run.smithi086.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2024-03-22T19:00:04.213 INFO:teuthology.orchestra.run.smithi131.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 885 kB/s | 59 kB 00:00 2024-03-22T19:00:04.229 INFO:teuthology.orchestra.run.smithi086.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-03-22T19:00:04.314 INFO:teuthology.orchestra.run.smithi131.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 750 kB/s | 75 kB 00:00 2024-03-22T19:00:04.330 INFO:teuthology.orchestra.run.smithi086.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 234 kB/s | 29 kB 00:00 2024-03-22T19:00:04.355 INFO:teuthology.orchestra.run.smithi086.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2024-03-22T19:00:04.359 INFO:teuthology.orchestra.run.smithi167.stdout:(133/146): librados2-18.2.2-819.g899aeb57.el8.x 4.3 MB/s | 3.7 MB 00:00 2024-03-22T19:00:04.418 INFO:teuthology.orchestra.run.smithi167.stdout:(134/146): librbd1-18.2.2-819.g899aeb57.el8.x86 5.1 MB/s | 3.6 MB 00:00 2024-03-22T19:00:04.430 INFO:teuthology.orchestra.run.smithi086.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2024-03-22T19:00:04.439 INFO:teuthology.orchestra.run.smithi131.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2024-03-22T19:00:04.443 INFO:teuthology.orchestra.run.smithi167.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 885 kB/s | 237 kB 00:00 2024-03-22T19:00:04.456 INFO:teuthology.orchestra.run.smithi086.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2024-03-22T19:00:04.506 INFO:teuthology.orchestra.run.smithi131.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 656 kB/s | 44 kB 00:00 2024-03-22T19:00:04.510 INFO:teuthology.orchestra.run.smithi167.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2024-03-22T19:00:04.531 INFO:teuthology.orchestra.run.smithi086.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 612 kB/s | 61 kB 00:00 2024-03-22T19:00:04.535 INFO:teuthology.orchestra.run.smithi167.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 845 kB/s | 99 kB 00:00 2024-03-22T19:00:04.556 INFO:teuthology.orchestra.run.smithi086.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2024-03-22T19:00:04.598 INFO:teuthology.orchestra.run.smithi131.stdout:(42/146): ceph-mgr-diskprediction-local-18.2.2- 5.8 MB/s | 7.4 MB 00:01 2024-03-22T19:00:04.623 INFO:teuthology.orchestra.run.smithi086.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 250 kB/s | 23 kB 00:00 2024-03-22T19:00:04.623 INFO:teuthology.orchestra.run.smithi131.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2024-03-22T19:00:04.644 INFO:teuthology.orchestra.run.smithi167.stdout:(138/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2024-03-22T19:00:04.665 INFO:teuthology.orchestra.run.smithi086.stdout:(69/146): python3-kubernetes-11.0.0-6.el8.noarc 1.1 MB/s | 1.2 MB 00:01 2024-03-22T19:00:04.669 INFO:teuthology.orchestra.run.smithi167.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 646 kB/s | 103 kB 00:00 2024-03-22T19:00:04.690 INFO:teuthology.orchestra.run.smithi086.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00 2024-03-22T19:00:04.702 INFO:teuthology.orchestra.run.smithi167.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2024-03-22T19:00:04.707 INFO:teuthology.orchestra.run.smithi131.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2024-03-22T19:00:04.749 INFO:teuthology.orchestra.run.smithi131.stdout:(45/146): python3-cachetools-3.1.1-4.el8.noarch 219 kB/s | 33 kB 00:00 2024-03-22T19:00:04.791 INFO:teuthology.orchestra.run.smithi131.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-03-22T19:00:04.820 INFO:teuthology.orchestra.run.smithi167.stdout:(141/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2024-03-22T19:00:04.832 INFO:teuthology.orchestra.run.smithi086.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2024-03-22T19:00:04.845 INFO:teuthology.orchestra.run.smithi167.stdout:(142/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-03-22T19:00:04.874 INFO:teuthology.orchestra.run.smithi086.stdout:(72/146): thrift-0.13.0-2.el8.x86_64.rpm 5.5 MB/s | 1.7 MB 00:00 2024-03-22T19:00:04.899 INFO:teuthology.orchestra.run.smithi086.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 216 kB/s | 44 kB 00:00 2024-03-22T19:00:04.904 INFO:teuthology.orchestra.run.smithi167.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.5 MB/s | 711 kB 00:00 2024-03-22T19:00:04.979 INFO:teuthology.orchestra.run.smithi167.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-03-22T19:00:04.983 INFO:teuthology.orchestra.run.smithi086.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 731 kB/s | 110 kB 00:00 2024-03-22T19:00:05.005 INFO:teuthology.orchestra.run.smithi167.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-03-22T19:00:05.067 INFO:teuthology.orchestra.run.smithi086.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 783 kB/s | 126 kB 00:00 2024-03-22T19:00:05.092 INFO:teuthology.orchestra.run.smithi086.stdout:(76/146): python3-jsonpatch-1.21-2.el8.noarch.r 122 kB/s | 27 kB 00:00 2024-03-22T19:00:05.096 INFO:teuthology.orchestra.run.smithi167.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-03-22T19:00:05.099 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-03-22T19:00:05.100 INFO:teuthology.orchestra.run.smithi167.stdout:Total 13 MB/s | 212 MB 00:16 2024-03-22T19:00:05.126 INFO:teuthology.orchestra.run.smithi086.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.1 MB/s | 457 kB 00:00 2024-03-22T19:00:05.221 INFO:teuthology.orchestra.run.smithi086.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 4.4 MB/s | 688 kB 00:00 2024-03-22T19:00:05.250 INFO:teuthology.orchestra.run.smithi131.stdout:(47/146): ceph-test-18.2.2-819.g899aeb57.el8.x8 5.2 MB/s | 47 MB 00:09 2024-03-22T19:00:05.260 INFO:teuthology.orchestra.run.smithi086.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.9 MB/s | 320 kB 00:00 2024-03-22T19:00:05.275 INFO:teuthology.orchestra.run.smithi131.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 168 kB/s | 81 kB 00:00 2024-03-22T19:00:05.302 INFO:teuthology.orchestra.run.smithi086.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-03-22T19:00:05.325 INFO:teuthology.orchestra.run.smithi131.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 353 kB/s | 18 kB 00:00 2024-03-22T19:00:05.335 INFO:teuthology.orchestra.run.smithi086.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 342 kB/s | 39 kB 00:00 2024-03-22T19:00:05.351 INFO:teuthology.orchestra.run.smithi131.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-03-22T19:00:05.376 INFO:teuthology.orchestra.run.smithi131.stdout:(51/146): python3-cherrypy-18.4.0-1.el8.noarch. 614 kB/s | 384 kB 00:00 2024-03-22T19:00:05.402 INFO:teuthology.orchestra.run.smithi086.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2024-03-22T19:00:05.427 INFO:teuthology.orchestra.run.smithi086.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2024-03-22T19:00:05.453 INFO:teuthology.orchestra.run.smithi086.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 884 kB/s | 103 kB 00:00 2024-03-22T19:00:05.459 INFO:teuthology.orchestra.run.smithi131.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 705 kB/s | 59 kB 00:00 2024-03-22T19:00:05.518 INFO:teuthology.orchestra.run.smithi131.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2024-03-22T19:00:05.543 INFO:teuthology.orchestra.run.smithi131.stdout:(54/146): python3-logutils-0.3.5-11.el8.noarch. 253 kB/s | 49 kB 00:00 2024-03-22T19:00:05.595 INFO:teuthology.orchestra.run.smithi131.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 445 kB/s | 60 kB 00:00 2024-03-22T19:00:05.603 INFO:teuthology.orchestra.run.smithi086.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.0 MB/s | 538 kB 00:00 2024-03-22T19:00:05.677 INFO:teuthology.orchestra.run.smithi131.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2024-03-22T19:00:05.702 INFO:teuthology.orchestra.run.smithi131.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 98 kB/s | 16 kB 00:00 2024-03-22T19:00:05.719 INFO:teuthology.orchestra.run.smithi131.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 271 kB/s | 34 kB 00:00 2024-03-22T19:00:05.720 INFO:teuthology.orchestra.run.smithi086.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2024-03-22T19:00:05.779 INFO:teuthology.orchestra.run.smithi086.stdout:(87/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-03-22T19:00:05.803 INFO:teuthology.orchestra.run.smithi131.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-03-22T19:00:05.828 INFO:teuthology.orchestra.run.smithi131.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2024-03-22T19:00:05.845 INFO:teuthology.orchestra.run.smithi131.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 149 kB/s | 19 kB 00:00 2024-03-22T19:00:05.846 INFO:teuthology.orchestra.run.smithi086.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 9.5 MB/s | 3.7 MB 00:00 2024-03-22T19:00:05.870 INFO:teuthology.orchestra.run.smithi131.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2024-03-22T19:00:05.879 INFO:teuthology.orchestra.run.smithi086.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.4 MB/s | 225 kB 00:00 2024-03-22T19:00:05.896 INFO:teuthology.orchestra.run.smithi131.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2024-03-22T19:00:05.904 INFO:teuthology.orchestra.run.smithi086.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-03-22T19:00:05.912 INFO:teuthology.orchestra.run.smithi131.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 410 kB/s | 27 kB 00:00 2024-03-22T19:00:05.996 INFO:teuthology.orchestra.run.smithi086.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-03-22T19:00:06.004 INFO:teuthology.orchestra.run.smithi131.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2024-03-22T19:00:06.030 INFO:teuthology.orchestra.run.smithi131.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2024-03-22T19:00:06.063 INFO:teuthology.orchestra.run.smithi131.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 407 kB/s | 61 kB 00:00 2024-03-22T19:00:06.088 INFO:teuthology.orchestra.run.smithi131.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2024-03-22T19:00:06.114 INFO:teuthology.orchestra.run.smithi131.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 784 kB/s | 85 kB 00:00 2024-03-22T19:00:06.149 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-03-22T19:00:06.160 INFO:teuthology.orchestra.run.smithi086.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 17 MB/s | 5.2 MB 00:00 2024-03-22T19:00:06.173 INFO:teuthology.orchestra.run.smithi131.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 831 kB/s | 70 kB 00:00 2024-03-22T19:00:06.205 INFO:teuthology.orchestra.run.smithi086.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 3.3 MB/s | 714 kB 00:00 2024-03-22T19:00:06.280 INFO:teuthology.orchestra.run.smithi086.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-03-22T19:00:06.298 INFO:teuthology.orchestra.run.smithi131.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 160 kB/s | 20 kB 00:00 2024-03-22T19:00:06.306 INFO:teuthology.orchestra.run.smithi086.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2024-03-22T19:00:06.323 INFO:teuthology.orchestra.run.smithi131.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 231 kB/s | 48 kB 00:00 2024-03-22T19:00:06.339 INFO:teuthology.orchestra.run.smithi086.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-03-22T19:00:06.390 INFO:teuthology.orchestra.run.smithi131.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2024-03-22T19:00:06.406 INFO:teuthology.orchestra.run.smithi086.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 433 kB/s | 54 kB 00:00 2024-03-22T19:00:06.449 INFO:teuthology.orchestra.run.smithi131.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 879 kB/s | 110 kB 00:00 2024-03-22T19:00:06.456 INFO:teuthology.orchestra.run.smithi086.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-03-22T19:00:06.474 INFO:teuthology.orchestra.run.smithi131.stdout:(75/146): python3-prettytable-0.7.2-14.el8.noar 250 kB/s | 44 kB 00:00 2024-03-22T19:00:06.524 INFO:teuthology.orchestra.run.smithi131.stdout:(76/146): python3-jsonpatch-1.21-2.el8.noarch.r 198 kB/s | 27 kB 00:00 2024-03-22T19:00:06.544 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-03-22T19:00:06.544 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-03-22T19:00:06.583 INFO:teuthology.orchestra.run.smithi131.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 945 kB/s | 126 kB 00:00 2024-03-22T19:00:06.616 INFO:teuthology.orchestra.run.smithi131.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2024-03-22T19:00:06.640 INFO:teuthology.orchestra.run.smithi086.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 6.4 MB/s | 1.5 MB 00:00 2024-03-22T19:00:06.658 INFO:teuthology.orchestra.run.smithi131.stdout:(79/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2024-03-22T19:00:06.682 INFO:teuthology.orchestra.run.smithi086.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.5 MB/s | 812 kB 00:00 2024-03-22T19:00:06.700 INFO:teuthology.orchestra.run.smithi131.stdout:(80/146): flex-2.6.1-9.el8.x86_64.rpm 2.7 MB/s | 320 kB 00:00 2024-03-22T19:00:06.767 INFO:teuthology.orchestra.run.smithi131.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2024-03-22T19:00:06.826 INFO:teuthology.orchestra.run.smithi131.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-03-22T19:00:06.851 INFO:teuthology.orchestra.run.smithi131.stdout:(83/146): python3-markupsafe-0.23-19.el8.x86_64 202 kB/s | 39 kB 00:00 2024-03-22T19:00:06.882 INFO:teuthology.orchestra.run.smithi086.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 24 MB/s | 14 MB 00:00 2024-03-22T19:00:06.908 INFO:teuthology.orchestra.run.smithi086.stdout:(102/146): python3-packaging-16.8-9.el8.noarch. 214 kB/s | 48 kB 00:00 2024-03-22T19:00:06.910 INFO:teuthology.orchestra.run.smithi131.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2024-03-22T19:00:06.927 INFO:teuthology.orchestra.run.smithi131.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2024-03-22T19:00:07.027 INFO:teuthology.orchestra.run.smithi131.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2024-03-22T19:00:07.033 INFO:teuthology.orchestra.run.smithi086.stdout:(103/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-03-22T19:00:07.084 INFO:teuthology.orchestra.run.smithi086.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 89 kB/s | 18 kB 00:00 2024-03-22T19:00:07.109 INFO:teuthology.orchestra.run.smithi086.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 212 kB/s | 43 kB 00:00 2024-03-22T19:00:07.116 INFO:teuthology.orchestra.run.smithi131.stdout:(87/146): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-03-22T19:00:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2024-03-22T19:00:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 769 kB/s | 109 kB 00:00 2024-03-22T19:00:07.269 INFO:teuthology.orchestra.run.smithi131.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-03-22T19:00:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 255 kB/s | 43 kB 00:00 2024-03-22T19:00:07.294 INFO:teuthology.orchestra.run.smithi131.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.2 MB/s | 225 kB 00:00 2024-03-22T19:00:07.351 INFO:teuthology.orchestra.run.smithi086.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 106 kB/s | 19 kB 00:00 2024-03-22T19:00:07.361 INFO:teuthology.orchestra.run.smithi131.stdout:(90/146): python3-numpy-1.14.3-10.el8.x86_64.rp 8.6 MB/s | 3.7 MB 00:00 2024-03-22T19:00:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 262 kB/s | 44 kB 00:00 2024-03-22T19:00:07.403 INFO:teuthology.orchestra.run.smithi131.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2024-03-22T19:00:07.427 INFO:teuthology.orchestra.run.smithi086.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-03-22T19:00:07.462 INFO:teuthology.orchestra.run.smithi131.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2024-03-22T19:00:07.469 INFO:teuthology.orchestra.run.smithi086.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-03-22T19:00:07.494 INFO:teuthology.orchestra.run.smithi086.stdout:(113/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-03-22T19:00:07.553 INFO:teuthology.orchestra.run.smithi086.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.6 MB/s | 200 kB 00:00 2024-03-22T19:00:07.596 INFO:teuthology.orchestra.run.smithi131.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 17 MB/s | 5.2 MB 00:00 2024-03-22T19:00:07.628 INFO:teuthology.orchestra.run.smithi086.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 651 kB/s | 87 kB 00:00 2024-03-22T19:00:07.629 INFO:teuthology.orchestra.run.smithi131.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2024-03-22T19:00:07.645 INFO:teuthology.orchestra.run.smithi086.stdout:(116/146): keyutils-libs-devel-1.5.10-9.el8.x86 273 kB/s | 48 kB 00:00 2024-03-22T19:00:07.704 INFO:teuthology.orchestra.run.smithi086.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 383 kB/s | 58 kB 00:00 2024-03-22T19:00:07.729 INFO:teuthology.orchestra.run.smithi131.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-03-22T19:00:07.744 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-03-22T19:00:07.755 INFO:teuthology.orchestra.run.smithi131.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-03-22T19:00:07.755 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-03-22T19:00:07.762 INFO:teuthology.orchestra.run.smithi086.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 289 kB/s | 39 kB 00:00 2024-03-22T19:00:07.780 INFO:teuthology.orchestra.run.smithi131.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2024-03-22T19:00:07.796 INFO:teuthology.orchestra.run.smithi086.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 552 kB/s | 83 kB 00:00 2024-03-22T19:00:07.813 INFO:teuthology.orchestra.run.smithi131.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 648 kB/s | 54 kB 00:00 2024-03-22T19:00:07.888 INFO:teuthology.orchestra.run.smithi086.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-03-22T19:00:07.913 INFO:teuthology.orchestra.run.smithi086.stdout:(121/146): libuuid-devel-2.32.1-43.el8.x86_64.r 852 kB/s | 100 kB 00:00 2024-03-22T19:00:07.938 INFO:teuthology.orchestra.run.smithi086.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 981 kB/s | 172 kB 00:00 2024-03-22T19:00:07.972 INFO:teuthology.orchestra.run.smithi131.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 981 kB/s | 188 kB 00:00 2024-03-22T19:00:08.039 INFO:teuthology.orchestra.run.smithi086.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 580 kB/s | 73 kB 00:00 2024-03-22T19:00:08.098 INFO:teuthology.orchestra.run.smithi131.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 5.3 MB/s | 1.5 MB 00:00 2024-03-22T19:00:08.147 INFO:teuthology.orchestra.run.smithi086.stdout:(124/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 918 kB/s | 238 kB 00:00 2024-03-22T19:00:08.181 INFO:teuthology.orchestra.run.smithi086.stdout:(125/146): elfutils-libelf-devel-0.189-3.el8.x8 435 kB/s | 62 kB 00:00 2024-03-22T19:00:08.206 INFO:teuthology.orchestra.run.smithi086.stdout:(126/146): NetworkManager-initscripts-updown-1. 538 kB/s | 144 kB 00:00 2024-03-22T19:00:08.264 INFO:teuthology.orchestra.run.smithi131.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-03-22T19:00:08.315 INFO:teuthology.orchestra.run.smithi131.stdout:(102/146): postgresql-contrib-10.23-4.module+el 2.3 MB/s | 812 kB 00:00 2024-03-22T19:00:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 2.4 MB/s | 562 kB 00:00 2024-03-22T19:00:08.399 INFO:teuthology.orchestra.run.smithi086.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 245 kB/s | 47 kB 00:00 2024-03-22T19:00:08.407 INFO:teuthology.orchestra.run.smithi131.stdout:(103/146): postgresql-server-10.23-4.module+el8 16 MB/s | 5.1 MB 00:00 2024-03-22T19:00:08.432 INFO:teuthology.orchestra.run.smithi131.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 107 kB/s | 18 kB 00:00 2024-03-22T19:00:08.465 INFO:teuthology.orchestra.run.smithi131.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 283 kB/s | 43 kB 00:00 2024-03-22T19:00:08.491 INFO:teuthology.orchestra.run.smithi086.stdout:(129/146): openssl-devel-1.1.1k-12.el8_9.x86_64 7.5 MB/s | 2.3 MB 00:00 2024-03-22T19:00:08.524 INFO:teuthology.orchestra.run.smithi131.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.3 MB/s | 155 kB 00:00 2024-03-22T19:00:08.549 INFO:teuthology.orchestra.run.smithi131.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 932 kB/s | 109 kB 00:00 2024-03-22T19:00:08.558 INFO:teuthology.orchestra.run.smithi086.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 3.0 MB/s | 559 kB 00:00 2024-03-22T19:00:08.574 INFO:teuthology.orchestra.run.smithi131.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 392 kB/s | 43 kB 00:00 2024-03-22T19:00:08.600 INFO:teuthology.orchestra.run.smithi131.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 248 kB/s | 19 kB 00:00 2024-03-22T19:00:08.616 INFO:teuthology.orchestra.run.smithi086.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 3.9 MB/s | 232 kB 00:00 2024-03-22T19:00:08.658 INFO:teuthology.orchestra.run.smithi131.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 402 kB/s | 44 kB 00:00 2024-03-22T19:00:08.700 INFO:teuthology.orchestra.run.smithi131.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2024-03-22T19:00:08.767 INFO:teuthology.orchestra.run.smithi131.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-03-22T19:00:08.792 INFO:teuthology.orchestra.run.smithi131.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2024-03-22T19:00:08.799 INFO:teuthology.orchestra.run.smithi086.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.6 MB/s | 303 kB 00:00 2024-03-22T19:00:08.867 INFO:teuthology.orchestra.run.smithi131.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2024-03-22T19:00:08.942 INFO:teuthology.orchestra.run.smithi086.stdout:(133/146): librados2-18.2.2-819.g899aeb57.el8.x 6.7 MB/s | 3.7 MB 00:00 2024-03-22T19:00:08.943 INFO:teuthology.orchestra.run.smithi131.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 274 kB/s | 48 kB 00:00 2024-03-22T19:00:08.975 INFO:teuthology.orchestra.run.smithi086.stdout:(134/146): libmount-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 237 kB 00:00 2024-03-22T19:00:09.026 INFO:teuthology.orchestra.run.smithi131.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 364 kB/s | 58 kB 00:00 2024-03-22T19:00:09.052 INFO:teuthology.orchestra.run.smithi131.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 355 kB/s | 39 kB 00:00 2024-03-22T19:00:09.077 INFO:teuthology.orchestra.run.smithi131.stdout:(118/146): libsepol-devel-2.9-3.el8.x86_64.rpm 306 kB/s | 87 kB 00:00 2024-03-22T19:00:09.100 INFO:teuthology.orchestra.run.smithi086.stdout:(135/146): libuuid-2.32.1-43.el8.x86_64.rpm 793 kB/s | 99 kB 00:00 2024-03-22T19:00:09.126 INFO:teuthology.orchestra.run.smithi086.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 978 kB/s | 179 kB 00:00 2024-03-22T19:00:09.177 INFO:teuthology.orchestra.run.smithi131.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 551 kB/s | 83 kB 00:00 2024-03-22T19:00:09.193 INFO:teuthology.orchestra.run.smithi086.stdout:(137/146): librbd1-18.2.2-819.g899aeb57.el8.x86 5.1 MB/s | 3.6 MB 00:00 2024-03-22T19:00:09.252 INFO:teuthology.orchestra.run.smithi086.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 819 kB/s | 103 kB 00:00 2024-03-22T19:00:09.253 INFO:teuthology.orchestra.run.smithi131.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.1 MB/s | 645 kB 00:00 2024-03-22T19:00:09.302 INFO:teuthology.orchestra.run.smithi086.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2024-03-22T19:00:09.335 INFO:teuthology.orchestra.run.smithi086.stdout:(140/146): libblkid-2.32.1-43.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2024-03-22T19:00:09.336 INFO:teuthology.orchestra.run.smithi131.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2024-03-22T19:00:09.361 INFO:teuthology.orchestra.run.smithi131.stdout:(122/146): libuuid-devel-2.32.1-43.el8.x86_64.r 542 kB/s | 100 kB 00:00 2024-03-22T19:00:09.394 INFO:teuthology.orchestra.run.smithi086.stdout:(141/146): libfdisk-2.32.1-43.el8.x86_64.rpm 2.7 MB/s | 254 kB 00:00 2024-03-22T19:00:09.410 INFO:teuthology.orchestra.run.smithi131.stdout:(123/146): libquadmath-8.5.0-20.el8.x86_64.rpm 519 kB/s | 172 kB 00:00 2024-03-22T19:00:09.428 INFO:teuthology.orchestra.run.smithi086.stdout:(142/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.6 MB/s | 854 kB 00:00 2024-03-22T19:00:09.445 INFO:teuthology.orchestra.run.smithi131.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 667 kB/s | 73 kB 00:00 2024-03-22T19:00:09.470 INFO:teuthology.orchestra.run.smithi131.stdout:(125/146): NetworkManager-initscripts-updown-1. 1.3 MB/s | 144 kB 00:00 2024-03-22T19:00:09.496 INFO:teuthology.orchestra.run.smithi131.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 719 kB/s | 62 kB 00:00 2024-03-22T19:00:09.503 INFO:teuthology.orchestra.run.smithi086.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.2 MB/s | 711 kB 00:00 2024-03-22T19:00:09.536 INFO:teuthology.orchestra.run.smithi086.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 3.0 MB/s | 333 kB 00:00 2024-03-22T19:00:09.579 INFO:teuthology.orchestra.run.smithi131.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.1 MB/s | 562 kB 00:00 2024-03-22T19:00:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:(145/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 1.5 MB 00:00 2024-03-22T19:00:09.621 INFO:teuthology.orchestra.run.smithi131.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 376 kB/s | 47 kB 00:00 2024-03-22T19:00:09.653 INFO:teuthology.orchestra.run.smithi086.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-03-22T19:00:09.656 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-22T19:00:09.656 INFO:teuthology.orchestra.run.smithi086.stdout:Total 11 MB/s | 212 MB 00:18 2024-03-22T19:00:09.672 INFO:teuthology.orchestra.run.smithi131.stdout:(129/146): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-03-22T19:00:09.722 INFO:teuthology.orchestra.run.smithi131.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 3.8 MB/s | 559 kB 00:00 2024-03-22T19:00:09.897 INFO:teuthology.orchestra.run.smithi131.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2024-03-22T19:00:10.048 INFO:teuthology.orchestra.run.smithi131.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.0 MB/s | 303 kB 00:00 2024-03-22T19:00:10.181 INFO:teuthology.orchestra.run.smithi131.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 237 kB 00:00 2024-03-22T19:00:10.323 INFO:teuthology.orchestra.run.smithi131.stdout:(134/146): librbd1-18.2.2-819.g899aeb57.el8.x86 5.5 MB/s | 3.6 MB 00:00 2024-03-22T19:00:10.349 INFO:teuthology.orchestra.run.smithi131.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 179 kB 00:00 2024-03-22T19:00:10.432 INFO:teuthology.orchestra.run.smithi131.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 914 kB/s | 99 kB 00:00 2024-03-22T19:00:10.516 INFO:teuthology.orchestra.run.smithi131.stdout:(137/146): util-linux-2.32.1-43.el8.x86_64.rpm 15 MB/s | 2.5 MB 00:00 2024-03-22T19:00:10.599 INFO:teuthology.orchestra.run.smithi131.stdout:(138/146): librados2-18.2.2-819.g899aeb57.el8.x 3.7 MB/s | 3.7 MB 00:00 2024-03-22T19:00:10.624 INFO:teuthology.orchestra.run.smithi131.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 534 kB/s | 103 kB 00:00 2024-03-22T19:00:10.683 INFO:teuthology.orchestra.run.smithi131.stdout:(140/146): libfdisk-2.32.1-43.el8.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2024-03-22T19:00:10.716 INFO:teuthology.orchestra.run.smithi131.stdout:(141/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.2 MB/s | 854 kB 00:00 2024-03-22T19:00:10.750 INFO:teuthology.orchestra.run.smithi131.stdout:(142/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-22T19:00:10.842 INFO:teuthology.orchestra.run.smithi131.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2024-03-22T19:00:10.936 INFO:teuthology.orchestra.run.smithi131.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.7 MB/s | 1.5 MB 00:00 2024-03-22T19:00:10.967 INFO:teuthology.orchestra.run.smithi131.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.5 MB/s | 333 kB 00:00 2024-03-22T19:00:11.051 INFO:teuthology.orchestra.run.smithi131.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2024-03-22T19:00:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-03-22T19:00:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:Total 9.9 MB/s | 212 MB 00:21 2024-03-22T19:00:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-03-22T19:00:12.106 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-03-22T19:00:12.242 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-03-22T19:00:12.242 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-03-22T19:00:12.499 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-03-22T19:00:12.499 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-03-22T19:00:13.689 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-03-22T19:00:13.699 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-03-22T19:00:13.725 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-03-22T19:00:13.860 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-22T19:00:13.860 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-03-22T19:00:13.873 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-03-22T19:00:14.110 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-22T19:00:14.428 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-22T19:00:14.812 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-22T19:00:14.973 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-22T19:00:15.032 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-22T19:00:15.231 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-22T19:00:15.285 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-22T19:00:15.511 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-22T19:00:15.722 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-22T19:00:15.882 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-22T19:00:16.009 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-22T19:00:16.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-22T19:00:16.599 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-22T19:00:16.840 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:00:17.212 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:00:17.431 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:00:17.541 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:00:17.773 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:00:17.865 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:00:18.197 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 12/162 2024-03-22T19:00:18.489 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-22T19:00:18.572 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-22T19:00:18.812 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-22T19:00:19.093 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-22T19:00:19.284 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-22T19:00:19.476 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-03-22T19:00:19.598 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-03-22T19:00:19.676 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-22T19:00:19.688 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-22T19:00:19.734 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-22T19:00:19.917 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-22T19:00:19.976 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-22T19:00:20.005 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-22T19:00:20.156 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:00:20.202 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-22T19:00:20.208 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-22T19:00:20.258 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:00:20.393 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-22T19:00:20.395 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-22T19:00:20.431 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 20/162 2024-03-22T19:00:20.505 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 20/162 2024-03-22T19:00:20.590 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-22T19:00:20.626 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-22T19:00:20.676 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-22T19:00:20.700 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-22T19:00:20.703 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-22T19:00:20.851 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-22T19:00:20.881 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-22T19:00:20.948 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-22T19:00:20.964 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-22T19:00:20.980 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-22T19:00:21.148 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-22T19:00:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-22T19:00:21.269 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-22T19:00:21.368 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-22T19:00:21.430 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-22T19:00:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-22T19:00:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-22T19:00:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-22T19:00:21.611 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-22T19:00:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-22T19:00:21.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-22T19:00:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-22T19:00:21.817 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-22T19:00:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-22T19:00:22.219 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-22T19:00:22.402 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-22T19:00:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-22T19:00:22.459 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:00:22.597 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-22T19:00:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:00:22.843 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:00:23.034 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:00:23.075 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:00:23.163 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:00:23.182 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-22T19:00:23.297 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:00:23.384 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:00:23.397 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-22T19:00:23.430 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:00:23.475 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:00:23.623 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:00:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:00:23.809 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 12/162 2024-03-22T19:00:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-22T19:00:24.079 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-22T19:00:24.086 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 12/162 2024-03-22T19:00:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-22T19:00:24.361 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-22T19:00:24.390 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-22T19:00:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-22T19:00:24.511 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-22T19:00:24.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-22T19:00:24.647 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-22T19:00:24.748 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-22T19:00:24.828 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-22T19:00:24.840 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-22T19:00:24.995 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x 31/162 2024-03-22T19:00:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-22T19:00:25.157 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8 32/162 2024-03-22T19:00:25.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-22T19:00:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-22T19:00:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-22T19:00:25.459 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 33/162 2024-03-22T19:00:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:00:25.663 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-22T19:00:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-22T19:00:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:00:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 20/162 2024-03-22T19:00:25.893 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-22T19:00:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 20/162 2024-03-22T19:00:26.072 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:00:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-22T19:00:26.201 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:00:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-22T19:00:26.364 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 20/162 2024-03-22T19:00:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-22T19:00:26.498 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 20/162 2024-03-22T19:00:26.680 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-22T19:00:26.832 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-22T19:00:27.016 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-22T19:00:27.039 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-22T19:00:27.216 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-22T19:00:27.415 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-22T19:00:27.723 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-22T19:00:27.815 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-22T19:00:27.822 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-22T19:00:27.913 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-22T19:00:28.000 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-22T19:00:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-22T19:00:28.694 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-22T19:00:28.738 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-22T19:00:28.794 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-22T19:00:28.939 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-22T19:00:29.097 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-22T19:00:29.161 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-22T19:00:29.262 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-22T19:00:29.372 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-22T19:00:29.559 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-22T19:00:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-22T19:00:29.609 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-22T19:00:29.829 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-22T19:00:29.864 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-22T19:00:29.882 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-22T19:00:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-22T19:00:30.093 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-22T19:00:30.128 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 41/162 2024-03-22T19:00:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-22T19:00:30.248 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x 31/162 2024-03-22T19:00:30.296 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 42/162 2024-03-22T19:00:30.385 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8 32/162 2024-03-22T19:00:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-22T19:00:30.557 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-22T19:00:30.656 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 33/162 2024-03-22T19:00:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-22T19:00:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-22T19:00:30.929 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-22T19:00:31.011 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-22T19:00:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-22T19:00:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x 31/162 2024-03-22T19:00:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-22T19:00:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8 32/162 2024-03-22T19:00:31.457 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-22T19:00:31.620 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-22T19:00:31.651 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 33/162 2024-03-22T19:00:31.662 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-22T19:00:31.937 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-22T19:00:32.091 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-22T19:00:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-22T19:00:32.409 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-22T19:00:32.454 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-22T19:00:32.482 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-22T19:00:32.685 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-22T19:00:32.687 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-22T19:00:32.748 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-22T19:00:33.022 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-22T19:00:33.031 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-22T19:00:33.263 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-22T19:00:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-22T19:00:33.429 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-22T19:00:33.527 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-22T19:00:33.529 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-22T19:00:33.703 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-22T19:00:33.904 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-22T19:00:34.138 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-22T19:00:34.149 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-22T19:00:34.323 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-03-22T19:00:34.500 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-22T19:00:34.575 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-03-22T19:00:34.711 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-22T19:00:34.836 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-03-22T19:00:34.917 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-22T19:00:35.024 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-03-22T19:00:35.151 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 41/162 2024-03-22T19:00:35.255 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-03-22T19:00:35.336 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 42/162 2024-03-22T19:00:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-22T19:00:35.588 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-22T19:00:35.649 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-22T19:00:35.824 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-03-22T19:00:35.868 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-22T19:00:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-22T19:00:35.942 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-22T19:00:36.147 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-22T19:00:36.157 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-03-22T19:00:36.171 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-22T19:00:36.401 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-22T19:00:36.402 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-22T19:00:36.459 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-03-22T19:00:36.590 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-22T19:00:36.618 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 41/162 2024-03-22T19:00:36.675 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-03-22T19:00:36.827 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 42/162 2024-03-22T19:00:36.883 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-03-22T19:00:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-22T19:00:37.078 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-22T19:00:37.131 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-03-22T19:00:37.186 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-22T19:00:37.382 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-22T19:00:37.400 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-22T19:00:37.429 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-22T19:00:37.462 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-03-22T19:00:37.501 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-22T19:00:37.627 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-22T19:00:37.738 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-22T19:00:37.782 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-03-22T19:00:37.845 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-22T19:00:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-22T19:00:38.019 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-22T19:00:38.114 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-03-22T19:00:38.210 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-22T19:00:38.212 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-22T19:00:38.361 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-03-22T19:00:38.557 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-22T19:00:38.580 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-03-22T19:00:38.783 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-22T19:00:38.866 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-03-22T19:00:39.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-22T19:00:39.134 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-03-22T19:00:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-22T19:00:39.171 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-22T19:00:39.294 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-22T19:00:39.322 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-03-22T19:00:39.350 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-22T19:00:39.495 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-22T19:00:39.547 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-03-22T19:00:39.573 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-22T19:00:39.693 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-22T19:00:39.770 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-22T19:00:39.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-03-22T19:00:39.892 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-22T19:00:40.001 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-22T19:00:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-03-22T19:00:40.201 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-22T19:00:40.212 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-22T19:00:40.266 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-22T19:00:40.441 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-22T19:00:40.546 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-22T19:00:40.616 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-22T19:00:40.769 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-22T19:00:40.803 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-22T19:00:40.889 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-22T19:00:41.035 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-22T19:00:41.042 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-22T19:00:41.089 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-22T19:00:41.189 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-22T19:00:41.241 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-22T19:00:41.291 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-22T19:00:41.406 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-22T19:00:41.448 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-22T19:00:41.526 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-22T19:00:41.630 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-22T19:00:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-22T19:00:41.844 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-22T19:00:41.849 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-22T19:00:42.040 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-22T19:00:42.105 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-22T19:00:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-22T19:00:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-22T19:00:42.348 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-22T19:00:42.430 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-22T19:00:42.492 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-22T19:00:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-22T19:00:42.667 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-22T19:00:42.765 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-22T19:00:42.779 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-22T19:00:42.864 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-22T19:00:42.947 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-22T19:00:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-22T19:00:43.006 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-22T19:00:43.225 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-22T19:00:43.274 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-22T19:00:43.275 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-22T19:00:43.465 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-22T19:00:43.474 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-22T19:00:43.537 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-22T19:00:43.672 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-22T19:00:43.710 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-22T19:00:43.741 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-22T19:00:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-22T19:00:43.927 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-22T19:00:43.929 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-22T19:00:43.964 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-22T19:00:44.129 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-22T19:00:44.203 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-22T19:00:44.245 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-22T19:00:44.349 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-22T19:00:44.492 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-22T19:00:44.549 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-22T19:00:44.702 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-03-22T19:00:44.739 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-22T19:00:44.853 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-22T19:00:44.894 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-22T19:00:44.951 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-22T19:00:45.219 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-22T19:00:45.298 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-22T19:00:45.299 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-03-22T19:00:45.454 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-22T19:00:45.565 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-03-22T19:00:45.610 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-22T19:00:45.647 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-22T19:00:45.761 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-03-22T19:00:45.824 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-22T19:00:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-03-22T19:00:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-22T19:00:46.015 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-22T19:00:46.086 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-03-22T19:00:46.243 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-22T19:00:46.270 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-03-22T19:00:46.336 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-22T19:00:46.466 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-22T19:00:46.497 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-03-22T19:00:46.595 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-22T19:00:46.667 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-22T19:00:46.680 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-22T19:00:46.810 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-22T19:00:46.819 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-22T19:00:46.870 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-22T19:00:47.059 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-22T19:00:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-22T19:00:47.251 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-22T19:00:47.427 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-22T19:00:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-22T19:00:47.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-22T19:00:47.625 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-22T19:00:47.658 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-22T19:00:47.777 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 103/162 2024-03-22T19:00:47.825 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-22T19:00:47.834 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 103/162 2024-03-22T19:00:47.937 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-22T19:00:48.019 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 104/162 2024-03-22T19:00:48.069 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-22T19:00:48.087 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-22T19:00:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-22T19:00:48.337 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-22T19:00:48.494 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-22T19:00:48.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-22T19:00:48.852 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-22T19:00:49.060 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-22T19:00:49.276 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-22T19:00:49.441 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-22T19:00:49.443 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-22T19:00:49.723 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-22T19:00:49.766 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-22T19:00:49.961 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-22T19:00:49.970 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-22T19:00:50.188 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-22T19:00:50.296 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-22T19:00:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-22T19:00:50.572 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-22T19:00:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-22T19:00:51.587 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-22T19:00:51.688 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:00:51.779 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 103/162 2024-03-22T19:00:51.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-22T19:00:51.844 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 103/162 2024-03-22T19:00:51.944 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:00:52.055 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 104/162 2024-03-22T19:00:52.122 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-22T19:00:52.373 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-22T19:00:52.524 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:00:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-22T19:00:52.838 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 106/162 2024-03-22T19:00:52.921 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 106/162 2024-03-22T19:00:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-22T19:00:53.093 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 107/162 2024-03-22T19:00:53.236 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-22T19:00:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-22T19:00:54.243 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 103/162 2024-03-22T19:00:54.349 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 103/162 2024-03-22T19:00:55.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 104/162 2024-03-22T19:00:55.758 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:00:56.004 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:00:56.584 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:00:56.883 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 106/162 2024-03-22T19:00:57.009 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 106/162 2024-03-22T19:00:57.162 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 107/162 2024-03-22T19:01:00.046 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:01:00.961 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:01:01.681 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 105/162 2024-03-22T19:01:02.012 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 106/162 2024-03-22T19:01:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 106/162 2024-03-22T19:01:02.343 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 107/162 2024-03-22T19:01:12.631 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 107/162 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2024-03-22T19:01:12.632 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T19:01:13.023 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 108/162 2024-03-22T19:01:14.594 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 108/162 2024-03-22T19:01:14.776 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 109/162 2024-03-22T19:01:14.997 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 109/162 2024-03-22T19:01:15.174 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 110/162 2024-03-22T19:01:15.631 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 110/162 2024-03-22T19:01:15.815 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 111/162 2024-03-22T19:01:15.832 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 111/162 2024-03-22T19:01:16.477 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 107/162 2024-03-22T19:01:16.477 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-03-22T19:01:16.477 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /run 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2024-03-22T19:01:16.478 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T19:01:16.859 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 108/162 2024-03-22T19:01:17.639 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:18.2.2-819.g899aeb57.el8.noarch 112/162 2024-03-22T19:01:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:18.2.2-819.g899aeb57.el8.noarch 112/162 2024-03-22T19:01:17.946 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el 113/162 2024-03-22T19:01:18.156 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.e 114/162 2024-03-22T19:01:18.389 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 115/162 2024-03-22T19:01:18.424 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 108/162 2024-03-22T19:01:18.612 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 115/162 2024-03-22T19:01:18.646 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 109/162 2024-03-22T19:01:18.866 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 109/162 2024-03-22T19:01:18.950 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 116/162 2024-03-22T19:01:19.059 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 110/162 2024-03-22T19:01:19.515 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 110/162 2024-03-22T19:01:19.717 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 111/162 2024-03-22T19:01:19.735 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 111/162 2024-03-22T19:01:19.919 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 116/162 2024-03-22T19:01:20.122 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 117/162 2024-03-22T19:01:20.217 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 117/162 2024-03-22T19:01:20.621 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8 118/162 2024-03-22T19:01:20.842 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 119/162 2024-03-22T19:01:20.881 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 119/162 2024-03-22T19:01:21.055 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 120/162 2024-03-22T19:01:21.067 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 120/162 2024-03-22T19:01:21.542 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:18.2.2-819.g899aeb57.el8.noarch 112/162 2024-03-22T19:01:21.731 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:18.2.2-819.g899aeb57.el8.noarch 112/162 2024-03-22T19:01:21.847 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el 113/162 2024-03-22T19:01:22.060 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.e 114/162 2024-03-22T19:01:22.233 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 115/162 2024-03-22T19:01:22.334 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:18.2.2-819.g899aeb57.el8.x86_64 121/162 2024-03-22T19:01:22.457 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 115/162 2024-03-22T19:01:22.550 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 122/162 2024-03-22T19:01:22.585 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 122/162 2024-03-22T19:01:22.778 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 116/162 2024-03-22T19:01:22.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-819.g899aeb 123/162 2024-03-22T19:01:22.824 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-819.g899aeb 123/162 2024-03-22T19:01:23.014 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 124/162 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 107/162 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /run 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2024-03-22T19:01:23.164 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:01:23.362 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 124/162 2024-03-22T19:01:23.530 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 125/162 2024-03-22T19:01:23.590 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 108/162 2024-03-22T19:01:23.741 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 116/162 2024-03-22T19:01:23.924 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 117/162 2024-03-22T19:01:24.019 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 117/162 2024-03-22T19:01:24.382 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8 118/162 2024-03-22T19:01:24.595 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 119/162 2024-03-22T19:01:24.634 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 119/162 2024-03-22T19:01:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 120/162 2024-03-22T19:01:24.811 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 120/162 2024-03-22T19:01:25.287 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 108/162 2024-03-22T19:01:25.500 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 109/162 2024-03-22T19:01:25.766 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 109/162 2024-03-22T19:01:25.965 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 110/162 2024-03-22T19:01:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:18.2.2-819.g899aeb57.el8.x86_64 121/162 2024-03-22T19:01:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 122/162 2024-03-22T19:01:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 122/162 2024-03-22T19:01:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-819.g899aeb 123/162 2024-03-22T19:01:26.483 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 110/162 2024-03-22T19:01:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-819.g899aeb 123/162 2024-03-22T19:01:26.640 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 111/162 2024-03-22T19:01:26.681 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 111/162 2024-03-22T19:01:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 124/162 2024-03-22T19:01:27.089 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 124/162 2024-03-22T19:01:27.282 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 125/162 2024-03-22T19:01:27.963 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 125/162 2024-03-22T19:01:28.954 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 126/162 2024-03-22T19:01:29.292 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-22T19:01:29.508 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:18.2.2-819.g899aeb57.el8.noarch 112/162 2024-03-22T19:01:29.595 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-22T19:01:29.661 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:18.2.2-819.g899aeb57.el8.noarch 112/162 2024-03-22T19:01:29.789 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el 113/162 2024-03-22T19:01:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.e 114/162 2024-03-22T19:01:29.992 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 129/162 2024-03-22T19:01:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 115/162 2024-03-22T19:01:30.356 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-22T19:01:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 115/162 2024-03-22T19:01:30.604 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-22T19:01:30.617 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-22T19:01:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 116/162 2024-03-22T19:01:30.823 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-22T19:01:31.025 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-22T19:01:31.238 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_6 134/162 2024-03-22T19:01:31.702 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 125/162 2024-03-22T19:01:31.800 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 116/162 2024-03-22T19:01:31.972 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 117/162 2024-03-22T19:01:32.102 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 117/162 2024-03-22T19:01:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 126/162 2024-03-22T19:01:32.505 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8 118/162 2024-03-22T19:01:32.533 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-22T19:01:32.675 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 119/162 2024-03-22T19:01:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 119/162 2024-03-22T19:01:32.756 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-22T19:01:32.881 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 120/162 2024-03-22T19:01:32.909 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 120/162 2024-03-22T19:01:33.447 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-22T19:01:33.543 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:33.744 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-22T19:01:33.863 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:33.901 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:34.030 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-22T19:01:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:18.2.2-819.g899aeb57.el8.x86_64 121/162 2024-03-22T19:01:34.262 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 131/162 2024-03-22T19:01:34.346 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-22T19:01:34.424 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 122/162 2024-03-22T19:01:34.476 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-22T19:01:34.487 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 122/162 2024-03-22T19:01:34.677 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-22T19:01:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-819.g899aeb 123/162 2024-03-22T19:01:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-819.g899aeb 123/162 2024-03-22T19:01:34.778 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-22T19:01:34.832 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_6 134/162 2024-03-22T19:01:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 124/162 2024-03-22T19:01:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-22T19:01:35.187 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 139/162 2024-03-22T19:01:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 124/162 2024-03-22T19:01:35.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 140/162 2024-03-22T19:01:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 125/162 2024-03-22T19:01:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 141/162 2024-03-22T19:01:35.999 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-22T19:01:36.236 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-22T19:01:36.315 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-22T19:01:36.545 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-22T19:01:36.558 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-22T19:01:36.749 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-22T19:01:36.759 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-22T19:01:36.897 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-22T19:01:37.102 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-22T19:01:37.145 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:37.280 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-22T19:01:37.490 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:37.499 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-22T19:01:37.499 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-22T19:01:37.527 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:37.790 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-22T19:01:37.790 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-22T19:01:37.955 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-22T19:01:38.040 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-22T19:01:38.040 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-22T19:01:38.287 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-22T19:01:38.318 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-22T19:01:38.551 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-22T19:01:38.571 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-22T19:01:38.730 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-22T19:01:38.738 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 139/162 2024-03-22T19:01:38.972 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 140/162 2024-03-22T19:01:39.005 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-22T19:01:39.005 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-22T19:01:39.258 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 141/162 2024-03-22T19:01:39.281 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-22T19:01:39.281 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-22T19:01:39.499 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-22T19:01:39.539 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-22T19:01:39.539 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-22T19:01:39.689 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-22T19:01:39.698 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-22T19:01:39.839 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-22T19:01:39.839 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-22T19:01:39.858 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-22T19:01:40.088 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-22T19:01:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-22T19:01:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-22T19:01:40.164 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-22T19:01:40.349 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 125/162 2024-03-22T19:01:40.388 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-22T19:01:40.402 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-22T19:01:40.405 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-22T19:01:40.579 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-22T19:01:40.605 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-22T19:01:40.606 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-22T19:01:40.790 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-22T19:01:40.902 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 126/162 2024-03-22T19:01:40.942 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-22T19:01:40.942 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-22T19:01:41.216 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-22T19:01:41.217 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-22T19:01:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-22T19:01:41.434 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-22T19:01:41.434 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-22T19:01:41.513 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:18.2.2-819.g899aeb57.el8.x86_64 1/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 2/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 3/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 4/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-819.g899aeb 5/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 6/162 2024-03-22T19:01:41.555 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 7/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 8/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 12/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 13/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_6 14/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 15/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 16/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 17/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 18/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8 20/162 2024-03-22T19:01:41.556 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x 21/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 22/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 23/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 24/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 25/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 26/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 27/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 28/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.e 29/162 2024-03-22T19:01:41.557 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 30/162 2024-03-22T19:01:41.569 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 31/162 2024-03-22T19:01:41.569 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 32/162 2024-03-22T19:01:41.569 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8 33/162 2024-03-22T19:01:41.569 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 34/162 2024-03-22T19:01:41.569 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el 35/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 36/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:18.2.2-819.g899aeb57.el8.noarch 37/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 38/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-22T19:01:41.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-22T19:01:41.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-03-22T19:01:41.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-03-22T19:01:41.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 93/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 95/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 96/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 97/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 98/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-22T19:01:41.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-22T19:01:41.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-22T19:01:41.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:18.2.2-819.g899aeb57.el8.x86_64 131/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 133/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-22T19:01:41.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-22T19:01:41.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-22T19:01:41.579 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-22T19:01:41.579 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-22T19:01:41.579 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-22T19:01:41.579 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-22T19:01:41.695 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-22T19:01:41.932 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-22T19:01:42.124 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-22T19:01:42.365 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-22T19:01:42.365 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-22T19:01:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-22T19:01:42.599 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-22T19:01:42.599 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-22T19:01:42.801 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-22T19:01:42.866 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-22T19:01:42.866 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-22T19:01:43.101 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-22T19:01:43.116 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-22T19:01:43.116 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-22T19:01:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 131/162 2024-03-22T19:01:43.349 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-22T19:01:43.349 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-22T19:01:43.592 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-22T19:01:43.599 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-22T19:01:43.782 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-22T19:01:43.782 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-22T19:01:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-22T19:01:44.052 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_6 134/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:18.2.2-819.g899aeb57.el8.x86_64 1/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 2/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 3/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 4/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-819.g899aeb 5/162 2024-03-22T19:01:44.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 6/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 7/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 8/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 12/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 13/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_6 14/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 15/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 16/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 17/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 18/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:01:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8 20/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x 21/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 22/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 23/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 24/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 25/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 26/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 27/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 28/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.e 29/162 2024-03-22T19:01:44.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 30/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 31/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 32/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8 33/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 34/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el 35/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 36/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:18.2.2-819.g899aeb57.el8.noarch 37/162 2024-03-22T19:01:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 38/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-22T19:01:44.775 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-22T19:01:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-22T19:01:44.777 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-22T19:01:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-22T19:01:44.779 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-22T19:01:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:18.2.2-819.g899aeb57.el8.x86_64 131/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 133/162 2024-03-22T19:01:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-22T19:01:44.782 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-22T19:01:44.783 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-22T19:01:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:47.788 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:47.883 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-22T19:01:48.282 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-22T19:01:48.670 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-22T19:01:48.925 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-22T19:01:49.113 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 139/162 2024-03-22T19:01:49.371 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 140/162 2024-03-22T19:01:49.693 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 141/162 2024-03-22T19:01:49.964 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-22T19:01:50.128 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-22T19:01:50.153 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-22T19:01:50.366 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-22T19:01:50.561 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-22T19:01:50.676 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-22T19:01:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-22T19:01:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-22T19:01:51.151 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-22T19:01:51.355 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-22T19:01:51.532 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-22T19:01:51.532 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-22T19:01:51.866 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-22T19:01:51.866 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-22T19:01:52.151 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-22T19:01:52.151 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-22T19:01:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-22T19:01:52.411 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-22T19:01:52.412 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2024-03-22T19:01:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-22T19:01:52.863 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.013 INFO:teuthology.orchestra.run.smithi167.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-22T19:01:53.014 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.1-8.el8.noarch 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.0.4-10.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.015 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-22T19:01:53.016 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.1-9.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.6-7.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-22T19:01:53.017 INFO:teuthology.orchestra.run.smithi167.stdout: libpq-13.11-1.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-22T19:01:53.018 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-22T19:01:53.019 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-22T19:01:53.020 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-22T19:01:53.021 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-22T19:01:53.022 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-22T19:01:53.023 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-22T19:01:53.024 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-22T19:01:53.025 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T19:01:53.025 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-03-22T19:01:53.138 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-22T19:01:53.138 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-22T19:01:53.219 DEBUG:teuthology.parallel:result is None 2024-03-22T19:01:53.406 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-22T19:01:53.406 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-22T19:01:53.723 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-22T19:01:53.723 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-22T19:01:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-22T19:01:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-22T19:01:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-22T19:01:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-22T19:01:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-22T19:01:54.731 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-22T19:01:54.731 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-22T19:01:55.355 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-22T19:01:55.355 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-22T19:01:55.355 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:01:56.012 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:18.2.2-819.g899aeb57.el8.x86_64 1/162 2024-03-22T19:01:56.012 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 2/162 2024-03-22T19:01:56.012 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 3/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 4/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-819.g899aeb 5/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 6/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 7/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 8/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 9/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 10/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 11/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 12/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 13/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_6 14/162 2024-03-22T19:01:56.013 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 15/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 16/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 17/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_ 18/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 19/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8 20/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x 21/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 22/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 23/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 24/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 25/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 26/162 2024-03-22T19:01:56.014 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 27/162 2024-03-22T19:01:56.026 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 28/162 2024-03-22T19:01:56.026 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.e 29/162 2024-03-22T19:01:56.026 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noar 30/162 2024-03-22T19:01:56.026 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.no 31/162 2024-03-22T19:01:56.026 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-819.g899a 32/162 2024-03-22T19:01:56.026 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8 33/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 34/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el 35/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 36/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:18.2.2-819.g899aeb57.el8.noarch 37/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 38/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-22T19:01:56.027 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-22T19:01:56.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-22T19:01:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-03-22T19:01:56.030 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-03-22T19:01:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-22T19:01:56.032 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-22T19:01:56.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:18.2.2-819.g899aeb57.el8.x86_64 131/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 133/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-22T19:01:56.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-22T19:01:56.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-22T19:01:56.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-22T19:01:56.320 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-22T19:01:56.321 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.924 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.1-8.el8.noarch 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.0.4-10.el8.x86_64 2024-03-22T19:01:56.925 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.926 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-22T19:01:56.927 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.1-9.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: jq-1.6-7.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libpq-13.11-1.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-22T19:01:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-22T19:01:56.929 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:01:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-22T19:01:56.931 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-22T19:01:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-2.6-1.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-22T19:01:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-22T19:01:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-22T19:01:56.935 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-22T19:01:56.936 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-22T19:01:56.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-03-22T19:01:56.936 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-03-22T19:01:57.136 DEBUG:teuthology.parallel:result is None 2024-03-22T19:02:11.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-22T19:02:11.617 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2024-03-22T19:02:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:02:12.260 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-03-22T19:02:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-22T19:02:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.1-8.el8.noarch 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.0.4-10.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:18.2.2-819.g899aeb57.el8.noarch 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.1-9.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.6-7.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libpq-13.11-1.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-22T19:02:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-22T19:02:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-22T19:02:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-22T19:02:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-2.6-1.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-22T19:02:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-22T19:02:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:18.2.2-819.g899aeb57.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-22T19:02:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-22T19:02:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-22T19:02:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:02:12.272 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-22T19:02:12.616 DEBUG:teuthology.parallel:result is None 2024-03-22T19:02:12.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:02:12.836 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T19:02:12.888 INFO:teuthology.orchestra.run.smithi086.stdout:18.2.2-819.g899aeb57.el8 2024-03-22T19:02:12.889 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-819.g899aeb57.el8 2024-03-22T19:02:12.889 INFO:teuthology.task.install:The correct ceph version 18.2.2-819.g899aeb57 is installed. 2024-03-22T19:02:12.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:02:13.112 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T19:02:13.141 INFO:teuthology.orchestra.run.smithi131.stdout:18.2.2-819.g899aeb57.el8 2024-03-22T19:02:13.141 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-819.g899aeb57.el8 2024-03-22T19:02:13.141 INFO:teuthology.task.install:The correct ceph version 18.2.2-819.g899aeb57 is installed. 2024-03-22T19:02:13.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:02:13.361 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T19:02:13.391 INFO:teuthology.orchestra.run.smithi167.stdout:18.2.2-819.g899aeb57.el8 2024-03-22T19:02:13.391 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-819.g899aeb57.el8 2024-03-22T19:02:13.391 INFO:teuthology.task.install:The correct ceph version 18.2.2-819.g899aeb57 is installed. 2024-03-22T19:02:13.393 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-22T19:02:13.393 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:02:13.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T19:02:13.479 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:02:13.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T19:02:13.528 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:02:13.528 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T19:02:13.577 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-22T19:02:13.577 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:02:13.577 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T19:02:13.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T19:02:13.744 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:02:13.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T19:02:13.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T19:02:13.847 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:02:13.847 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T19:02:13.880 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T19:02:13.951 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-22T19:02:13.951 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:02:13.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T19:02:14.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T19:02:14.120 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:02:14.120 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T19:02:14.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T19:02:14.220 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:02:14.220 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T19:02:14.253 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T19:02:14.323 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-22T19:02:14.324 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:02:14.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T19:02:14.388 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T19:02:14.493 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:02:14.493 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T19:02:14.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T19:02:14.595 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:02:14.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T19:02:14.627 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T19:02:14.698 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-22T19:02:14.783 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46', 'subvols': {'create': 2}} 2024-03-22T19:02:14.783 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:02:14.784 INFO:tasks.cephadm:Cluster fsid is b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:14.784 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-22T19:02:14.784 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.131', 'mon.c': '172.21.15.167'} 2024-03-22T19:02:14.784 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-03-22T19:02:14.784 INFO:tasks.cephadm:First mgr is x 2024-03-22T19:02:14.784 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-22T19:02:14.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-03-22T19:02:14.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-03-22T19:02:14.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-03-22T19:02:14.915 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-22T19:02:14.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:02:15.056 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/77886/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-819-g899aeb57', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-819.g899aeb57'}, 'url': 'https://3.chacra.ceph.com/r/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 15:40:31.893897', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-yuri-RunCephCmd-base-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-22T19:02:15.273 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref rishabh-yuri-RunCephCmd-base-reef, sha1 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:02:15.275 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/x86_64/flavors/default/cephadm 2024-03-22T19:02:15.275 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:15.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702392 Mar 22 19:02 /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:15.712 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.118 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702392 Mar 22 19:02 /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.119 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-yuri-RunCephCmd-base-reef/899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.511 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702392 Mar 22 19:02 /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.512 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.561 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.583 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T19:02:16.619 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 on all hosts... 2024-03-22T19:02:16.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 pull 2024-03-22T19:02:16.624 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 pull 2024-03-22T19:02:16.626 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 pull 2024-03-22T19:02:17.564 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46... 2024-03-22T19:02:17.817 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46... 2024-03-22T19:02:18.360 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46... 2024-03-22T19:02:43.270 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-03-22T19:02:43.270 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.2.2-819-g899aeb57 (899aeb57f4ac58b35d5b52546466a4c7cb3bdc46) reef (stable)", 2024-03-22T19:02:43.270 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "b3d9fcfc8fbba7a8f65ff96d4a9dc5372900253434097bdf6fa03e7fa85da57b", 2024-03-22T19:02:43.270 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-03-22T19:02:43.270 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6fbfc94e399c2d21d5693d138b1e3dd3a637b835d28c9772808e6afaafc4d94d" 2024-03-22T19:02:43.271 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-03-22T19:02:43.271 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-03-22T19:02:46.043 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:02:46.043 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.2.2-819-g899aeb57 (899aeb57f4ac58b35d5b52546466a4c7cb3bdc46) reef (stable)", 2024-03-22T19:02:46.043 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "b3d9fcfc8fbba7a8f65ff96d4a9dc5372900253434097bdf6fa03e7fa85da57b", 2024-03-22T19:02:46.043 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-03-22T19:02:46.043 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6fbfc94e399c2d21d5693d138b1e3dd3a637b835d28c9772808e6afaafc4d94d" 2024-03-22T19:02:46.044 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:02:46.044 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 18.2.2-819-g899aeb57 (899aeb57f4ac58b35d5b52546466a4c7cb3bdc46) reef (stable)", 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "b3d9fcfc8fbba7a8f65ff96d4a9dc5372900253434097bdf6fa03e7fa85da57b", 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6fbfc94e399c2d21d5693d138b1e3dd3a637b835d28c9772808e6afaafc4d94d" 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-03-22T19:02:47.473 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-03-22T19:02:47.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-03-22T19:02:47.569 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-03-22T19:02:47.605 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-03-22T19:02:47.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-03-22T19:02:47.704 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-03-22T19:02:47.736 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-03-22T19:02:47.763 INFO:tasks.cephadm:Writing seed config... 2024-03-22T19:02:47.764 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-22T19:02:47.764 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-22T19:02:47.765 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-22T19:02:47.766 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-22T19:02:47.767 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-22T19:02:47.768 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-22T19:02:47.768 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-22T19:02:47.768 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:02:47.768 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-22T19:02:47.810 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b22c63ac-e87e-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-22T19:02:47.810 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a.service 2024-03-22T19:02:47.854 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x.service 2024-03-22T19:02:47.898 INFO:tasks.cephadm:Bootstrapping... 2024-03-22T19:02:47.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 -v bootstrap --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:02:47.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:02:47.962 INFO:journalctl@ceph.mgr.x.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:02:48.113 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-22T19:02:48.113 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46', '-v', 'bootstrap', '--fsid', 'b22c63ac-e87e-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-03-22T19:02:48.157 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.6.1 2024-03-22T19:02:48.158 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-22T19:02:48.158 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-03-22T19:02:48.203 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.6.1 2024-03-22T19:02:48.204 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-03-22T19:02:48.204 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-03-22T19:02:48.217 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T19:02:48.217 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T19:02:48.229 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T19:02:48.230 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-22T19:02:48.242 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-03-22T19:02:48.254 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-03-22T19:02:48.254 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-03-22T19:02:48.254 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-03-22T19:02:48.299 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.6.1 2024-03-22T19:02:48.299 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 4.6.1 is present 2024-03-22T19:02:48.300 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-03-22T19:02:48.300 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-03-22T19:02:48.312 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T19:02:48.312 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T19:02:48.324 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T19:02:48.324 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-22T19:02:48.337 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 140699908844008 on /run/cephadm/b22c63ac-e87e-11ee-95cd-87774f69a715.lock 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 140699908844008 acquired on /run/cephadm/b22c63ac-e87e-11ee-95cd-87774f69a715.lock 2024-03-22T19:02:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-03-22T19:02:48.350 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-03-22T19:02:48.350 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-03-22T19:02:48.358 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-03-22T19:02:48.358 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-03-22T19:02:48.365 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-22T19:02:48.365 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-22T19:02:48.365 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1758sec hoplimit 64 pref medium 2024-03-22T19:02:48.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-22T19:02:48.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-22T19:02:48.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T19:02:48.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-22T19:02:48.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T19:02:48.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T19:02:48.372 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-03-22T19:02:48.372 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-03-22T19:02:48.372 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-22T19:02:48.373 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-22T19:02:48.374 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46... 2024-03-22T19:02:48.780 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout b3d9fcfc8fbba7a8f65ff96d4a9dc5372900253434097bdf6fa03e7fa85da57b 2024-03-22T19:02:48.780 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46... 2024-03-22T19:02:48.780 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-03-22T19:02:48.780 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:b4f9a57f9170437205470e6e8737955fa9818ae41d0622bc4a9282059f5b9171 2024-03-22T19:02:48.780 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-22T19:02:48.780 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:b3d9fcfc8fbba7a8f65ff96d4a9dc5372900253434097bdf6fa03e7fa85da57b 2024-03-22T19:02:48.781 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-22T19:02:49.592 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 18.2.2-819-g899aeb57 (899aeb57f4ac58b35d5b52546466a4c7cb3bdc46) reef (stable) 2024-03-22T19:02:49.593 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 18.2.2-819-g899aeb57 (899aeb57f4ac58b35d5b52546466a4c7cb3bdc46) reef (stable) 2024-03-22T19:02:49.593 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-03-22T19:02:50.186 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-03-22T19:02:50.186 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-03-22T19:02:50.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQDa1f1lTvbcHRAAm8mGdWULBJ9fn7unCp0Itw== 2024-03-22T19:02:51.496 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQDb1f1lSwiCCBAAJpmz2+0bwI93SlHnzZJaaQ== 2024-03-22T19:02:52.144 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQDb1f1lrow9LxAAZtn6G0j2Ors6v8Mitjp1uA== 2024-03-22T19:02:52.146 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-03-22T19:02:52.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T19:02:52.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-22T19:02:52.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:52.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T19:02:52.794 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T19:02:52.794 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = pacific 2024-03-22T19:02:52.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:52.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T19:02:52.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:02:52.795 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-03-22T19:02:53.483 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-03-22T19:02:53.716 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-22T19:02:53.914 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-22T19:02:54.136 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715.target. 2024-03-22T19:02:54.136 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715.target. 2024-03-22T19:02:54.459 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a 2024-03-22T19:02:54.459 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-22T19:02:54.644 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715.target.wants/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@.service. 2024-03-22T19:02:54.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:54 smithi086 systemd[1]: Starting Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:02:55.171 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T19:02:55.171 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-22T19:02:55.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:55 smithi086 bash[37504]: 6933ebfcda08e1a7e9aaf1889cdb1a58010f06aa430fbb9da10b351f59b69804 2024-03-22T19:02:55.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:55 smithi086 systemd[1]: Started Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:02:55.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:55 smithi086 ceph-mon[37541]: mkfs b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:55.187 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T19:02:55.187 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-22T19:02:55.187 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-22T19:02:55.187 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T19:02:55.188 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-03-22T19:02:55.188 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-03-22T19:02:55.420 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:55 smithi086 ceph-mon[37541]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-22T19:02:56.094 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.587217s) 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-22T19:02:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.669+0000 7f3f2fe00700 1 Processor -- start 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.669+0000 7f3f2fe00700 1 -- start start 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.669+0000 7f3f2fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f300791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.669+0000 7f3f2fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300797a0 con 0x7f3f3007ad70 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.670+0000 7f3f2ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f300791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.670+0000 7f3f2ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f300791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34866/0 (socket says 172.21.15.86:34866) 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.670+0000 7f3f2ea00700 1 -- 172.21.15.86:0/1653104691 learned_addr learned my addr 172.21.15.86:0/1653104691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:02:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.670+0000 7f3f2ea00700 1 -- 172.21.15.86:0/1653104691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300798e0 con 0x7f3f3007ad70 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2ea00700 1 --2- 172.21.15.86:0/1653104691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f300791d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f200098a0 tx=0x7f3f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83447ed7feb784a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2d600700 1 -- 172.21.15.86:0/1653104691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f20004160 con 0x7f3f3007ad70 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2d600700 1 -- 172.21.15.86:0/1653104691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f200042c0 con 0x7f3f3007ad70 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2d600700 1 -- 172.21.15.86:0/1653104691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f200045a0 con 0x7f3f3007ad70 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2fe00700 1 -- 172.21.15.86:0/1653104691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 msgr2=0x7f3f300791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2fe00700 1 --2- 172.21.15.86:0/1653104691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f300791d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f200098a0 tx=0x7f3f20009bb0 comp rx=0 tx=0).stop 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2fe00700 1 -- 172.21.15.86:0/1653104691 shutdown_connections 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.671+0000 7f3f2fe00700 1 --2- 172.21.15.86:0/1653104691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f300791d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.672+0000 7f3f2fe00700 1 -- 172.21.15.86:0/1653104691 >> 172.21.15.86:0/1653104691 conn(0x7f3f301018c0 msgr2=0x7f3f30103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.672+0000 7f3f2fe00700 1 -- 172.21.15.86:0/1653104691 shutdown_connections 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.672+0000 7f3f2fe00700 1 -- 172.21.15.86:0/1653104691 wait complete. 2024-03-22T19:02:56.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.673+0000 7f3f2fe00700 1 Processor -- start 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.673+0000 7f3f2fe00700 1 -- start start 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.674+0000 7f3f2fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f3019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.674+0000 7f3f2fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3019bfc0 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.674+0000 7f3f2ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f3019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.674+0000 7f3f2ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f3019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34874/0 (socket says 172.21.15.86:34874) 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.674+0000 7f3f2ea00700 1 -- 172.21.15.86:0/759172629 learned_addr learned my addr 172.21.15.86:0/759172629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f2ea00700 1 -- 172.21.15.86:0/759172629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009550 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f2ea00700 1 --2- 172.21.15.86:0/759172629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f3019ba80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3f20004000 tx=0x7f3f20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f20011630 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20011790 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f20011a30 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f2fe00700 1 -- 172.21.15.86:0/759172629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3019c1c0 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.675+0000 7f3f2fe00700 1 -- 172.21.15.86:0/759172629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3019c660 con 0x7f3f3007ad70 2024-03-22T19:02:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.676+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3f20022070 con 0x7f3f3007ad70 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.676+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f2001c450 con 0x7f3f3007ad70 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.676+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f30079c50 con 0x7f3f3007ad70 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.679+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f3f2001b450 con 0x7f3f3007ad70 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.752+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3f30061de0 con 0x7f3f3007ad70 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.752+0000 7f3f27400700 1 -- 172.21.15.86:0/759172629 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f3f2001b5f0 con 0x7f3f3007ad70 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.753+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 msgr2=0x7f3f3019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.753+0000 7f3f24c00700 1 --2- 172.21.15.86:0/759172629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f3019ba80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3f20004000 tx=0x7f3f20004750 comp rx=0 tx=0).stop 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.753+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 shutdown_connections 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.753+0000 7f3f24c00700 1 --2- 172.21.15.86:0/759172629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3007ad70 0x7f3f3019ba80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.753+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 >> 172.21.15.86:0/759172629 conn(0x7f3f301018c0 msgr2=0x7f3f30103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.754+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 shutdown_connections 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:55.754+0000 7f3f24c00700 1 -- 172.21.15.86:0/759172629 wait complete. 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-03-22T19:02:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-03-22T19:02:56.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:56 smithi086 ceph-mon[37541]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-22T19:02:56.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:56 smithi086 ceph-mon[37541]: monmap e1: 1 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:02:56.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:56 smithi086 ceph-mon[37541]: fsmap 2024-03-22T19:02:56.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:56 smithi086 ceph-mon[37541]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T19:02:56.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:56 smithi086 ceph-mon[37541]: mgrmap e1: no daemons active 2024-03-22T19:02:56.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:56 smithi086 ceph-mon[37541]: from='client.? 172.21.15.86:0/759172629' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-22T19:02:58.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:57 smithi086 ceph-mon[37541]: from='client.? 172.21.15.86:0/3803159316' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T19:02:58.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:57 smithi086 ceph-mon[37541]: from='client.? 172.21.15.86:0/3803159316' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T19:02:58.129 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.611+0000 7f6adf600700 1 Processor -- start 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.611+0000 7f6adf600700 1 -- start start 2024-03-22T19:02:58.130 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.612+0000 7f6adf600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad8106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.612+0000 7f6adf600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8106f00 con 0x7f6ad8106510 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.612+0000 7f6adcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad8106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.612+0000 7f6adcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad8106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34882/0 (socket says 172.21.15.86:34882) 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.612+0000 7f6adcc00700 1 -- 172.21.15.86:0/2200264003 learned_addr learned my addr 172.21.15.86:0/2200264003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.612+0000 7f6adcc00700 1 -- 172.21.15.86:0/2200264003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8107710 con 0x7f6ad8106510 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.613+0000 7f6adcc00700 1 --2- 172.21.15.86:0/2200264003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad8106930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0098a0 tx=0x7f6acc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7578e69f52f80959 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.613+0000 7f6ad7400700 1 -- 172.21.15.86:0/2200264003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6acc004160 con 0x7f6ad8106510 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.613+0000 7f6ad7400700 1 -- 172.21.15.86:0/2200264003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc0042c0 con 0x7f6ad8106510 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.613+0000 7f6ad7400700 1 -- 172.21.15.86:0/2200264003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6acc0045a0 con 0x7f6ad8106510 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.613+0000 7f6adf600700 1 -- 172.21.15.86:0/2200264003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 msgr2=0x7f6ad8106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.613+0000 7f6adf600700 1 --2- 172.21.15.86:0/2200264003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad8106930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0098a0 tx=0x7f6acc009bb0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.614+0000 7f6adf600700 1 -- 172.21.15.86:0/2200264003 shutdown_connections 2024-03-22T19:02:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.614+0000 7f6adf600700 1 --2- 172.21.15.86:0/2200264003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad8106930 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.614+0000 7f6adf600700 1 -- 172.21.15.86:0/2200264003 >> 172.21.15.86:0/2200264003 conn(0x7f6ad8101ab0 msgr2=0x7f6ad8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.614+0000 7f6adf600700 1 -- 172.21.15.86:0/2200264003 shutdown_connections 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.614+0000 7f6adf600700 1 -- 172.21.15.86:0/2200264003 wait complete. 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adf600700 1 Processor -- start 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adf600700 1 -- start start 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adf600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adf600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad819c240 con 0x7f6ad8106510 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34894/0 (socket says 172.21.15.86:34894) 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.615+0000 7f6adcc00700 1 -- 172.21.15.86:0/3803159316 learned_addr learned my addr 172.21.15.86:0/3803159316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6adcc00700 1 -- 172.21.15.86:0/3803159316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc009550 con 0x7f6ad8106510 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6adcc00700 1 --2- 172.21.15.86:0/3803159316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad819bd00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6acc004000 tx=0x7f6acc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:02:58.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6acc011630 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc011790 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad819c440 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6acc011a30 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad819c8e0 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.616+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6acc021070 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.617+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6acc01b440 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.617+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad8107a80 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.620+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f6acc01a460 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.687+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6ad8107c10 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.691+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6acc01a600 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.691+0000 7f6ad5600700 1 -- 172.21.15.86:0/3803159316 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+454 (secure 0 0 0) 0x7f6acc0355b0 con 0x7f6ad8106510 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.692+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 msgr2=0x7f6ad819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:02:58.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.692+0000 7f6adf600700 1 --2- 172.21.15.86:0/3803159316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad819bd00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6acc004000 tx=0x7f6acc004750 comp rx=0 tx=0).stop 2024-03-22T19:02:58.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.692+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 shutdown_connections 2024-03-22T19:02:58.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.692+0000 7f6adf600700 1 --2- 172.21.15.86:0/3803159316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8106510 0x7f6ad819bd00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.692+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 >> 172.21.15.86:0/3803159316 conn(0x7f6ad8101ab0 msgr2=0x7f6ad807a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:02:58.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.693+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 shutdown_connections 2024-03-22T19:02:58.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:57.693+0000 7f6adf600700 1 -- 172.21.15.86:0/3803159316 wait complete. 2024-03-22T19:02:58.134 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-03-22T19:02:58.952 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:58 smithi086 ceph-mon[37541]: from='client.? 172.21.15.86:0/2038539294' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:02:58.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.544+0000 7f079ca00700 1 Processor -- start 2024-03-22T19:02:58.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.544+0000 7f079ca00700 1 -- start start 2024-03-22T19:02:58.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.544+0000 7f079ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f07980791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:02:58.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.544+0000 7f079ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798079780 con 0x7f079807ad50 2024-03-22T19:02:58.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0795e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f07980791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:02:58.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0795e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f07980791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37046/0 (socket says 172.21.15.86:37046) 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0795e00700 1 -- 172.21.15.86:0/3486657297 learned_addr learned my addr 172.21.15.86:0/3486657297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0795e00700 1 -- 172.21.15.86:0/3486657297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980798c0 con 0x7f079807ad50 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0795e00700 1 --2- 172.21.15.86:0/3486657297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f07980791b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f07880098a0 tx=0x7f0788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54b9407749b5769 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0794a00700 1 -- 172.21.15.86:0/3486657297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0788004160 con 0x7f079807ad50 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0794a00700 1 -- 172.21.15.86:0/3486657297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f07880042c0 con 0x7f079807ad50 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.545+0000 7f0794a00700 1 -- 172.21.15.86:0/3486657297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0788011a00 con 0x7f079807ad50 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 -- 172.21.15.86:0/3486657297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 msgr2=0x7f07980791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 --2- 172.21.15.86:0/3486657297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f07980791b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f07880098a0 tx=0x7f0788009bb0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 -- 172.21.15.86:0/3486657297 shutdown_connections 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 --2- 172.21.15.86:0/3486657297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f07980791b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 -- 172.21.15.86:0/3486657297 >> 172.21.15.86:0/3486657297 conn(0x7f0798101a50 msgr2=0x7f0798103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:02:58.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 -- 172.21.15.86:0/3486657297 shutdown_connections 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.546+0000 7f079ca00700 1 -- 172.21.15.86:0/3486657297 wait complete. 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.547+0000 7f079ca00700 1 Processor -- start 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.547+0000 7f079ca00700 1 -- start start 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f079ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f079819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f079ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079819c990 con 0x7f079807ad50 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f0795e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f079819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f0795e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f079819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37050/0 (socket says 172.21.15.86:37050) 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f0795e00700 1 -- 172.21.15.86:0/2038539294 learned_addr learned my addr 172.21.15.86:0/2038539294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f0795e00700 1 -- 172.21.15.86:0/2038539294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0788009550 con 0x7f079807ad50 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.548+0000 7f0795e00700 1 --2- 172.21.15.86:0/2038539294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f079819c450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0788009130 tx=0x7f0788011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0788003b70 con 0x7f079807ad50 2024-03-22T19:02:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0788003cd0 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079819cb90 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07880197d0 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079819d030 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f07880199c0 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.549+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f078800b720 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.550+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079810c8e0 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.553+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f078802a060 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.621+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f079810caa0 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.621+0000 7f078ea00700 1 -- 172.21.15.86:0/2038539294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f0788019db0 con 0x7f079807ad50 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.622+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 msgr2=0x7f079819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:02:58.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.622+0000 7f079ca00700 1 --2- 172.21.15.86:0/2038539294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f079819c450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0788009130 tx=0x7f0788011ad0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.622+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 shutdown_connections 2024-03-22T19:02:58.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.622+0000 7f079ca00700 1 --2- 172.21.15.86:0/2038539294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079807ad50 0x7f079819c450 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:02:58.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.622+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 >> 172.21.15.86:0/2038539294 conn(0x7f0798101a50 msgr2=0x7f0798102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:02:58.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.623+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 shutdown_connections 2024-03-22T19:02:58.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:02:58.623+0000 7f079ca00700 1 -- 172.21.15.86:0/2038539294 wait complete. 2024-03-22T19:02:58.961 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-03-22T19:02:59.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:58 smithi086 systemd[1]: Stopping Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:02:59.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:59 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a[37518]: 2024-03-22T19:02:59.267+0000 7fd72b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:02:59.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:59 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a[37518]: 2024-03-22T19:02:59.267+0000 7fd72b200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-22T19:02:59.909 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:59 smithi086 bash[37871]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a 2024-03-22T19:03:00.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:59 smithi086 systemd[1]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a.service: Succeeded. 2024-03-22T19:03:00.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:59 smithi086 systemd[1]: Stopped Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:03:00.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:02:59 smithi086 systemd[1]: Starting Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:03:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-22T19:03:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 bash[37994]: 0e4d0f4d2b91d500615575d4a20cb47611f83aacda82d0a35f5d9540c472fa97 2024-03-22T19:03:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 systemd[1]: Started Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:03:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T19:03:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: ceph version 18.2.2-819-g899aeb57 (899aeb57f4ac58b35d5b52546466a4c7cb3bdc46) reef (stable), process ceph-mon, pid 7 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: pidfile_write: ignore empty --pid-file 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: load: jerasure load: lrc 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: RocksDB version: 7.9.2 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Git sha 0 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Compile date 2024-03-22 14:25:53 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: DB SUMMARY 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: DB Session ID: M1D2VNBKLHMJJEMQ0WSI 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: CURRENT file: CURRENT 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-22T19:03:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98803 ; 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.error_if_exists: 0 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.create_if_missing: 0 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.paranoid_checks: 1 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.env: 0x5556328c1720 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.info_log: 0x5556355d3520 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.statistics: (nil) 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.use_fsync: 0 2024-03-22T19:03:00.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_log_file_size: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.allow_fallocate: 1 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.use_direct_reads: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T19:03:00.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.db_log_dir: 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.wal_dir: 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T19:03:00.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T19:03:00.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T19:03:00.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T19:03:00.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.write_buffer_manager: 0x55563486c320 2024-03-22T19:03:00.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T19:03:00.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T19:03:00.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.unordered_write: 0 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.row_cache: None 2024-03-22T19:03:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.wal_filter: None 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.two_write_queues: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.wal_compression: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.atomic_flush: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.log_readahead_size: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T19:03:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_background_jobs: 2 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_background_compactions: -1 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_subcompactions: 1 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T19:03:00.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_open_files: -1 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_background_flushes: -1 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Compression algorithms supported: 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kZSTD supported: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kXpressCompression supported: 0 2024-03-22T19:03:00.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kZlibCompression supported: 1 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kSnappyCompression supported: 1 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kLZ4Compression supported: 1 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: kBZip2Compression supported: 0 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.merge_operator: 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_filter: None 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T19:03:00.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5556355d3620) 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-03-22T19:03:00.771 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x5556348ed1f0 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-03-22T19:03:00.772 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-03-22T19:03:00.773 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression: NoCompression 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.num_levels: 7 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T19:03:00.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T19:03:00.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T19:03:00.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T19:03:00.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.table_properties_collectors: 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.inplace_update_support: 0 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T19:03:00.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.bloom_locality: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.max_successive_merges: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.ttl: 2592000 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.enable_blob_files: false 2024-03-22T19:03:00.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.min_blob_size: 0 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 85c7c7d6-48e5-4e8d-ab93-392e5dcb71ef 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711134180409819, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-22T19:03:00.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711134180412670, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94065, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 92060, "index_size": 287, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13646, "raw_average_key_size": 49, "raw_value_size": 84608, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711134180, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "85c7c7d6-48e5-4e8d-ab93-392e5dcb71ef", "db_session_id": "M1D2VNBKLHMJJEMQ0WSI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711134180412856, "job": 1, "event": "recovery_finished"} 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55563498a000 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: DB pointer 0x555634978000 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T19:03:00.781 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 93.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 53.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-22T19:03:00.782 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 93.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 53.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 53.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 53.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-22T19:03:00.783 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 10.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 10.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x5556348ed1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-22T19:03:00.784 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???) e1 preinit fsid b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).mds e1 new map 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).mds e1 print_map 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-22T19:03:00.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: monmap e1: 1 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: fsmap 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T19:03:00.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:00 smithi086 ceph-mon[38032]: mgrmap e1: no daemons active 2024-03-22T19:03:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.834+0000 7f04e2800700 1 Processor -- start 2024-03-22T19:03:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.834+0000 7f04e2800700 1 -- start start 2024-03-22T19:03:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.834+0000 7f04e2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc106740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.834+0000 7f04e2800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc106d10 con 0x7f04dc106320 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.835+0000 7f04e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.835+0000 7f04e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37062/0 (socket says 172.21.15.86:37062) 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.835+0000 7f04e1400700 1 -- 172.21.15.86:0/1294809220 learned_addr learned my addr 172.21.15.86:0/1294809220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.835+0000 7f04e1400700 1 -- 172.21.15.86:0/1294809220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc107520 con 0x7f04dc106320 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.836+0000 7f04e1400700 1 --2- 172.21.15.86:0/1294809220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc106740 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04d00098a0 tx=0x7f04d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456979128972c4a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.836+0000 7f04dbe00700 1 -- 172.21.15.86:0/1294809220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04d0004160 con 0x7f04dc106320 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.836+0000 7f04dbe00700 1 -- 172.21.15.86:0/1294809220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04d00042c0 con 0x7f04dc106320 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.836+0000 7f04dbe00700 1 -- 172.21.15.86:0/1294809220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04d0011a00 con 0x7f04dc106320 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.836+0000 7f04e2800700 1 -- 172.21.15.86:0/1294809220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 msgr2=0x7f04dc106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.836+0000 7f04e2800700 1 --2- 172.21.15.86:0/1294809220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc106740 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04d00098a0 tx=0x7f04d0009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.837+0000 7f04e2800700 1 -- 172.21.15.86:0/1294809220 shutdown_connections 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.837+0000 7f04e2800700 1 --2- 172.21.15.86:0/1294809220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc106740 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.837+0000 7f04e2800700 1 -- 172.21.15.86:0/1294809220 >> 172.21.15.86:0/1294809220 conn(0x7f04dc1018c0 msgr2=0x7f04dc103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.837+0000 7f04e2800700 1 -- 172.21.15.86:0/1294809220 shutdown_connections 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.837+0000 7f04e2800700 1 -- 172.21.15.86:0/1294809220 wait complete. 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e2800700 1 Processor -- start 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e2800700 1 -- start start 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e2800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc19c800 con 0x7f04dc106320 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37066/0 (socket says 172.21.15.86:37066) 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.838+0000 7f04e1400700 1 -- 172.21.15.86:0/2894784376 learned_addr learned my addr 172.21.15.86:0/2894784376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04e1400700 1 -- 172.21.15.86:0/2894784376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0009550 con 0x7f04dc106320 2024-03-22T19:03:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04e1400700 1 --2- 172.21.15.86:0/2894784376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc19c2c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009130 tx=0x7f04d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04d0003ab0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04d0003c10 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc19ca00 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04d00247d0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.839+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc19cea0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.840+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f04d0024d50 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.840+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f04d00193c0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.840+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc0623e0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.843+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f04d001abb0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.911+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f04dc19fbb0 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.917+0000 7f04da000700 1 -- 172.21.15.86:0/2894784376 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f04d001ad50 con 0x7f04dc106320 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 msgr2=0x7f04dc19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 --2- 172.21.15.86:0/2894784376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc19c2c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009130 tx=0x7f04d0011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 shutdown_connections 2024-03-22T19:03:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 --2- 172.21.15.86:0/2894784376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04dc106320 0x7f04dc19c2c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 >> 172.21.15.86:0/2894784376 conn(0x7f04dc1018c0 msgr2=0x7f04dc102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 shutdown_connections 2024-03-22T19:03:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:00.918+0000 7f04e2800700 1 -- 172.21.15.86:0/2894784376 wait complete. 2024-03-22T19:03:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-22T19:03:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:03:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-03-22T19:03:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-22T19:03:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-22T19:03:01.682 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x 2024-03-22T19:03:01.683 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-22T19:03:01.880 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715.target.wants/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@.service. 2024-03-22T19:03:02.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:01 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2894784376' entity='client.admin' 2024-03-22T19:03:02.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:01 smithi086 systemd[1]: Starting Ceph mgr.x for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:03:02.926 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T19:03:02.926 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-22T19:03:02.941 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T19:03:02.941 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-22T19:03:02.942 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-22T19:03:02.942 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T19:03:02.962 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T19:03:02.962 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-22T19:03:02.980 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T19:03:02.980 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-22T19:03:02.980 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-22T19:03:02.980 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-22T19:03:02.980 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-03-22T19:03:02.980 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-03-22T19:03:03.160 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:02 smithi086 bash[38267]: 104e8310a3b26b8f633d64e6075162be8e539e8e8e3d543ef6749de93c552788 2024-03-22T19:03:03.160 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:02 smithi086 systemd[1]: Started Ceph mgr.x for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:03:03.697 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:03 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/306481234' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T19:03:03.698 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:03 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:03.441+0000 7fa997455200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T19:03:03.698 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:03 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:03.576+0000 7fa997455200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T19:03:03.698 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:03 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:03.690+0000 7fa997455200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T19:03:03.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:03.859 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:03.859 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:03:03.859 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:03.860 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T19:03:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T19:03:03.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T19:03:03.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T19:03:03.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T19:03:03.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T19:03:03.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:03.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T19:02:55.168766+0000", 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.393+0000 7fdd98600700 1 Processor -- start 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.393+0000 7fdd98600700 1 -- start start 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.393+0000 7fdd98600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd90106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.393+0000 7fdd98600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90106ee0 con 0x7fdd901064f0 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.394+0000 7fdd95c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd90106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.394+0000 7fdd95c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd90106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37092/0 (socket says 172.21.15.86:37092) 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.394+0000 7fdd95c00700 1 -- 172.21.15.86:0/1551442716 learned_addr learned my addr 172.21.15.86:0/1551442716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.394+0000 7fdd95c00700 1 -- 172.21.15.86:0/1551442716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd901076f0 con 0x7fdd901064f0 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.394+0000 7fdd95c00700 1 --2- 172.21.15.86:0/1551442716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd90106910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098a0 tx=0x7fdd84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7a36413b1a00ee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.394+0000 7fdd8fe00700 1 -- 172.21.15.86:0/1551442716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd84004160 con 0x7fdd901064f0 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd8fe00700 1 -- 172.21.15.86:0/1551442716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd840042c0 con 0x7fdd901064f0 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd8fe00700 1 -- 172.21.15.86:0/1551442716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd84011a00 con 0x7fdd901064f0 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 -- 172.21.15.86:0/1551442716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 msgr2=0x7fdd90106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 --2- 172.21.15.86:0/1551442716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd90106910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098a0 tx=0x7fdd84009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 -- 172.21.15.86:0/1551442716 shutdown_connections 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 --2- 172.21.15.86:0/1551442716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd90106910 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 -- 172.21.15.86:0/1551442716 >> 172.21.15.86:0/1551442716 conn(0x7fdd90101a50 msgr2=0x7fdd90103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 -- 172.21.15.86:0/1551442716 shutdown_connections 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.395+0000 7fdd98600700 1 -- 172.21.15.86:0/1551442716 wait complete. 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.396+0000 7fdd98600700 1 Processor -- start 2024-03-22T19:03:03.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.396+0000 7fdd98600700 1 -- start start 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.397+0000 7fdd98600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd9019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.397+0000 7fdd98600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9019a7d0 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.397+0000 7fdd95c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd9019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.397+0000 7fdd95c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd9019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37100/0 (socket says 172.21.15.86:37100) 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.397+0000 7fdd95c00700 1 -- 172.21.15.86:0/306481234 learned_addr learned my addr 172.21.15.86:0/306481234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.397+0000 7fdd95c00700 1 -- 172.21.15.86:0/306481234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84009550 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd95c00700 1 --2- 172.21.15.86:0/306481234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd9019a290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009130 tx=0x7fdd84011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd84003ab0 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd84003c10 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd840247d0 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9019a9d0 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9019ae70 con 0x7fdd901064f0 2024-03-22T19:03:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdd84024d00 con 0x7fdd901064f0 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.398+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd840193c0 con 0x7fdd901064f0 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.399+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd900623e0 con 0x7fdd901064f0 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.402+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fdd8401abb0 con 0x7fdd901064f0 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.478+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdd9010a920 con 0x7fdd901064f0 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.479+0000 7fdd8e000700 1 -- 172.21.15.86:0/306481234 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fdd8401ad50 con 0x7fdd901064f0 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 msgr2=0x7fdd9019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 --2- 172.21.15.86:0/306481234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd9019a290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009130 tx=0x7fdd84011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 shutdown_connections 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 --2- 172.21.15.86:0/306481234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd901064f0 0x7fdd9019a290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 >> 172.21.15.86:0/306481234 conn(0x7fdd90101a50 msgr2=0x7fdd901026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 shutdown_connections 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:03.480+0000 7fdd98600700 1 -- 172.21.15.86:0/306481234 wait complete. 2024-03-22T19:03:03.875 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-03-22T19:03:04.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:03 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:03.802+0000 7fa997455200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T19:03:04.955 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:04 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:04.646+0000 7fa997455200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:03:05.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:04 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:04.953+0000 7fa997455200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T19:03:05.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:05.060+0000 7fa997455200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T19:03:05.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:05.196+0000 7fa997455200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T19:03:05.665 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:05.389+0000 7fa997455200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T19:03:05.666 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:05.554+0000 7fa997455200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T19:03:06.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:05.663+0000 7fa997455200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T19:03:06.419 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:06 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:06.158+0000 7fa997455200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T19:03:06.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:06 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2184447837' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T19:03:06.671 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:06 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:06.661+0000 7fa997455200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T19:03:06.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T19:03:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T19:03:06.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T19:03:06.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-22T19:03:06.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:06.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T19:03:06.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T19:03:06.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T19:03:06.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T19:03:06.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T19:03:06.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T19:03:06.835 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T19:03:06.836 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T19:02:55.168766+0000", 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.312+0000 7f18ca000700 1 Processor -- start 2024-03-22T19:03:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.313+0000 7f18ca000700 1 -- start start 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.313+0000 7f18ca000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c40795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.313+0000 7f18ca000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c407aec0 con 0x7f18c4079190 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.313+0000 7f18c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c40795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.313+0000 7f18c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c40795b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37114/0 (socket says 172.21.15.86:37114) 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.313+0000 7f18c3400700 1 -- 172.21.15.86:0/824619243 learned_addr learned my addr 172.21.15.86:0/824619243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.314+0000 7f18c3400700 1 -- 172.21.15.86:0/824619243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c407b000 con 0x7f18c4079190 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.314+0000 7f18c3400700 1 --2- 172.21.15.86:0/824619243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c40795b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18b40098a0 tx=0x7f18b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbba79fb505616 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.314+0000 7f18c2000700 1 -- 172.21.15.86:0/824619243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18b4004160 con 0x7f18c4079190 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.314+0000 7f18c2000700 1 -- 172.21.15.86:0/824619243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18b40042c0 con 0x7f18c4079190 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.314+0000 7f18c2000700 1 -- 172.21.15.86:0/824619243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18b4011a00 con 0x7f18c4079190 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 -- 172.21.15.86:0/824619243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 msgr2=0x7f18c40795b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 --2- 172.21.15.86:0/824619243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c40795b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18b40098a0 tx=0x7f18b4009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:06.838 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 -- 172.21.15.86:0/824619243 shutdown_connections 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 --2- 172.21.15.86:0/824619243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c40795b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 -- 172.21.15.86:0/824619243 >> 172.21.15.86:0/824619243 conn(0x7f18c4101aa0 msgr2=0x7f18c4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 -- 172.21.15.86:0/824619243 shutdown_connections 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.315+0000 7f18ca000700 1 -- 172.21.15.86:0/824619243 wait complete. 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.316+0000 7f18ca000700 1 Processor -- start 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.316+0000 7f18ca000700 1 -- start start 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.316+0000 7f18ca000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.316+0000 7f18ca000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419caa0 con 0x7f18c4079190 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.317+0000 7f18c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.317+0000 7f18c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37130/0 (socket says 172.21.15.86:37130) 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.317+0000 7f18c3400700 1 -- 172.21.15.86:0/2184447837 learned_addr learned my addr 172.21.15.86:0/2184447837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.317+0000 7f18c3400700 1 -- 172.21.15.86:0/2184447837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b4009550 con 0x7f18c4079190 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.317+0000 7f18c3400700 1 --2- 172.21.15.86:0/2184447837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c419c560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009130 tx=0x7f18b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:06.839 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.317+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18b4003b70 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.318+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18b4003cd0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.318+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c419cca0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.318+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18b40197d0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.318+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c419d140 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.319+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f18b401a430 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.319+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f18b4025be0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.319+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c40623e0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.321+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f18b402a060 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.396+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f18c419fac0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.397+0000 7f18bbe00700 1 -- 172.21.15.86:0/2184447837 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f18b401a8d0 con 0x7f18c4079190 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 msgr2=0x7f18c419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 --2- 172.21.15.86:0/2184447837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c419c560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009130 tx=0x7f18b4011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 shutdown_connections 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 --2- 172.21.15.86:0/2184447837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c4079190 0x7f18c419c560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:06.840 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 >> 172.21.15.86:0/2184447837 conn(0x7f18c4101aa0 msgr2=0x7f18c41026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 shutdown_connections 2024-03-22T19:03:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:06.398+0000 7f18ca000700 1 -- 172.21.15.86:0/2184447837 wait complete. 2024-03-22T19:03:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (2/15)... 2024-03-22T19:03:07.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:06 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:06.912+0000 7fa997455200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-22T19:03:07.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:07 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:07.032+0000 7fa997455200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T19:03:08.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:07 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:07.987+0000 7fa997455200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T19:03:08.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:08 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:08.118+0000 7fa997455200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T19:03:08.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:08 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:08.312+0000 7fa997455200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T19:03:09.416 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:09 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:09.165+0000 7fa997455200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-22T19:03:09.668 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:09 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:09.415+0000 7fa997455200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T19:03:10.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:09 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:09.667+0000 7fa997455200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T19:03:10.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:10.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1047902997' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T19:03:10.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:10.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:03:10.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T19:03:10.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T19:03:10.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T19:03:10.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T19:03:10.463 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.463 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T19:03:10.463 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:10.463 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T19:03:10.463 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T19:03:10.471 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T19:03:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T19:02:55.168766+0000", 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T19:03:10.473 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.019+0000 7efe09200700 1 Processor -- start 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.020+0000 7efe09200700 1 -- start start 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.020+0000 7efe09200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe040791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.020+0000 7efe09200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040797a0 con 0x7efe0407ad70 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.020+0000 7efe01e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe040791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.020+0000 7efe01e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe040791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46490/0 (socket says 172.21.15.86:46490) 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.020+0000 7efe01e00700 1 -- 172.21.15.86:0/2465394054 learned_addr learned my addr 172.21.15.86:0/2465394054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.021+0000 7efe01e00700 1 -- 172.21.15.86:0/2465394054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040798e0 con 0x7efe0407ad70 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.021+0000 7efe01e00700 1 --2- 172.21.15.86:0/2465394054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe040791d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098a0 tx=0x7efdf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0852d47cd1870 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.021+0000 7efe00a00700 1 -- 172.21.15.86:0/2465394054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf4004160 con 0x7efe0407ad70 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.021+0000 7efe00a00700 1 -- 172.21.15.86:0/2465394054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efdf40042c0 con 0x7efe0407ad70 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.021+0000 7efe00a00700 1 -- 172.21.15.86:0/2465394054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf4011a00 con 0x7efe0407ad70 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 -- 172.21.15.86:0/2465394054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 msgr2=0x7efe040791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:10.474 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 --2- 172.21.15.86:0/2465394054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe040791d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098a0 tx=0x7efdf4009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 -- 172.21.15.86:0/2465394054 shutdown_connections 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 --2- 172.21.15.86:0/2465394054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe040791d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 -- 172.21.15.86:0/2465394054 >> 172.21.15.86:0/2465394054 conn(0x7efe04101a50 msgr2=0x7efe04103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 -- 172.21.15.86:0/2465394054 shutdown_connections 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.022+0000 7efe09200700 1 -- 172.21.15.86:0/2465394054 wait complete. 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.023+0000 7efe09200700 1 Processor -- start 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.023+0000 7efe09200700 1 -- start start 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.023+0000 7efe09200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe0419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.024+0000 7efe09200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0419c9d0 con 0x7efe0407ad70 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.024+0000 7efe01e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe0419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.024+0000 7efe01e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe0419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46496/0 (socket says 172.21.15.86:46496) 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.024+0000 7efe01e00700 1 -- 172.21.15.86:0/1047902997 learned_addr learned my addr 172.21.15.86:0/1047902997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.024+0000 7efe01e00700 1 -- 172.21.15.86:0/1047902997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009550 con 0x7efe0407ad70 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.024+0000 7efe01e00700 1 --2- 172.21.15.86:0/1047902997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe0419c490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009130 tx=0x7efdf4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf4003ab0 con 0x7efe0407ad70 2024-03-22T19:03:10.475 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efdf4003c10 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0419cbd0 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf40247d0 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0419d070 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efdf4024a00 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.025+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efdf402db50 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.026+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0410c8c0 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.028+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7efdf402ddb0 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.103+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efe0410cab0 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.104+0000 7efdfaa00700 1 -- 172.21.15.86:0/1047902997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7efdf4035090 con 0x7efe0407ad70 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 msgr2=0x7efe0419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 --2- 172.21.15.86:0/1047902997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe0419c490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009130 tx=0x7efdf4011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 shutdown_connections 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 --2- 172.21.15.86:0/1047902997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0407ad70 0x7efe0419c490 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 >> 172.21.15.86:0/1047902997 conn(0x7efe04101a50 msgr2=0x7efe041026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:10.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 shutdown_connections 2024-03-22T19:03:10.477 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:10.105+0000 7efe09200700 1 -- 172.21.15.86:0/1047902997 wait complete. 2024-03-22T19:03:10.477 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (3/15)... 2024-03-22T19:03:11.217 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:10 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:10.849+0000 7fa997455200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T19:03:11.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:11.216+0000 7fa997455200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:03:11.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:11.332+0000 7fa997455200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T19:03:12.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:11.579+0000 7fa997455200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T19:03:12.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: Activating manager daemon x 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: mgrmap e2: x(active, starting, since 0.00437943s) 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: Manager daemon x is now available 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' 2024-03-22T19:03:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:11 smithi086 ceph-mon[38032]: from='mgr.14100 172.21.15.86:0/336853928' entity='mgr.x' 2024-03-22T19:03:13.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:13.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T19:03:13.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T19:03:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T19:03:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T19:03:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T19:03:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T19:03:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.582 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T19:02:55.168766+0000", 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-22T19:03:13.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.890+0000 7efca9e00700 1 Processor -- start 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.890+0000 7efca9e00700 1 -- start start 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.890+0000 7efca9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca40791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.890+0000 7efca9e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4079780 con 0x7efca407ad50 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.891+0000 7efca2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca40791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.891+0000 7efca2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca40791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46574/0 (socket says 172.21.15.86:46574) 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.891+0000 7efca2a00700 1 -- 172.21.15.86:0/1752659272 learned_addr learned my addr 172.21.15.86:0/1752659272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.891+0000 7efca2a00700 1 -- 172.21.15.86:0/1752659272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40798c0 con 0x7efca407ad50 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.891+0000 7efca2a00700 1 --2- 172.21.15.86:0/1752659272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca40791b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc940098a0 tx=0x7efc94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f325d8a3a6d0154e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca1600700 1 -- 172.21.15.86:0/1752659272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc94004160 con 0x7efca407ad50 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca1600700 1 -- 172.21.15.86:0/1752659272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc940042c0 con 0x7efca407ad50 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca1600700 1 -- 172.21.15.86:0/1752659272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc94011a00 con 0x7efca407ad50 2024-03-22T19:03:13.584 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 -- 172.21.15.86:0/1752659272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 msgr2=0x7efca40791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 --2- 172.21.15.86:0/1752659272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca40791b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc940098a0 tx=0x7efc94009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 -- 172.21.15.86:0/1752659272 shutdown_connections 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 --2- 172.21.15.86:0/1752659272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca40791b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 -- 172.21.15.86:0/1752659272 >> 172.21.15.86:0/1752659272 conn(0x7efca4101a50 msgr2=0x7efca4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 -- 172.21.15.86:0/1752659272 shutdown_connections 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.892+0000 7efca9e00700 1 -- 172.21.15.86:0/1752659272 wait complete. 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.893+0000 7efca9e00700 1 Processor -- start 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.893+0000 7efca9e00700 1 -- start start 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.894+0000 7efca9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.894+0000 7efca9e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca419c990 con 0x7efca407ad50 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.894+0000 7efca2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.894+0000 7efca2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46588/0 (socket says 172.21.15.86:46588) 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.894+0000 7efca2a00700 1 -- 172.21.15.86:0/1586760666 learned_addr learned my addr 172.21.15.86:0/1586760666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.894+0000 7efca2a00700 1 -- 172.21.15.86:0/1586760666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94009550 con 0x7efca407ad50 2024-03-22T19:03:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.895+0000 7efca2a00700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca419c450 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc94009130 tx=0x7efc94011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.895+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc94003b70 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.895+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc94003cd0 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.895+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc940197d0 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.895+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca419cb90 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.895+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca419d030 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.896+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 3) v1 ==== 45257+0+0 (secure 0 0 0) 0x7efc9401abb0 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.896+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca410c8e0 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.896+0000 7efca8a00700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7efc880383a0 0x7efc8803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.896+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efc9404c390 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.897+0000 7efca2000700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7efc880383a0 0x7efc8803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.897+0000 7efca2000700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7efc880383a0 0x7efc8803a860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc90006fd0 tx=0x7efc90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:12.902+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc9401a430 con 0x7efca407ad50 2024-03-22T19:03:13.586 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.180+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efca410cad0 con 0x7efca407ad50 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.180+0000 7efca8a00700 1 -- 172.21.15.86:0/1586760666 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7efc9402db40 con 0x7efca407ad50 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7efc880383a0 msgr2=0x7efc8803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7efc880383a0 0x7efc8803a860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc90006fd0 tx=0x7efc90006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 msgr2=0x7efca419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca419c450 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc94009130 tx=0x7efc94011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 shutdown_connections 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7efc880383a0 0x7efc8803a860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 --2- 172.21.15.86:0/1586760666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca407ad50 0x7efca419c450 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 >> 172.21.15.86:0/1586760666 conn(0x7efca4101a50 msgr2=0x7efca4102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 shutdown_connections 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.183+0000 7efca9e00700 1 -- 172.21.15.86:0/1586760666 wait complete. 2024-03-22T19:03:13.587 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-03-22T19:03:14.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:13 smithi086 ceph-mon[38032]: mgrmap e3: x(active, since 1.00911s) 2024-03-22T19:03:14.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1586760666' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T19:03:14.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-22T19:03:14.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.975+0000 7f70a8c00700 1 Processor -- start 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.975+0000 7f70a8c00700 1 -- start start 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.975+0000 7f70a8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a40791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.975+0000 7f70a8c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40797c0 con 0x7f70a407ad90 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.976+0000 7f70a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a40791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.976+0000 7f70a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a40791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46604/0 (socket says 172.21.15.86:46604) 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.976+0000 7f70a1e00700 1 -- 172.21.15.86:0/2385985747 learned_addr learned my addr 172.21.15.86:0/2385985747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.976+0000 7f70a1e00700 1 -- 172.21.15.86:0/2385985747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4079900 con 0x7f70a407ad90 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.976+0000 7f70a1e00700 1 --2- 172.21.15.86:0/2385985747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a40791f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f70940098a0 tx=0x7f7094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eb17efdb35c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a1400700 1 -- 172.21.15.86:0/2385985747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7094004160 con 0x7f70a407ad90 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a1400700 1 -- 172.21.15.86:0/2385985747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70940042c0 con 0x7f70a407ad90 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a1400700 1 -- 172.21.15.86:0/2385985747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7094011a00 con 0x7f70a407ad90 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a8c00700 1 -- 172.21.15.86:0/2385985747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 msgr2=0x7f70a40791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a8c00700 1 --2- 172.21.15.86:0/2385985747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a40791f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f70940098a0 tx=0x7f7094009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a8c00700 1 -- 172.21.15.86:0/2385985747 shutdown_connections 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a8c00700 1 --2- 172.21.15.86:0/2385985747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a40791f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.977+0000 7f70a8c00700 1 -- 172.21.15.86:0/2385985747 >> 172.21.15.86:0/2385985747 conn(0x7f70a4101a50 msgr2=0x7f70a4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.978+0000 7f70a8c00700 1 -- 172.21.15.86:0/2385985747 shutdown_connections 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.978+0000 7f70a8c00700 1 -- 172.21.15.86:0/2385985747 wait complete. 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.978+0000 7f70a8c00700 1 Processor -- start 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a8c00700 1 -- start start 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:14.544 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a8c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419c9d0 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46610/0 (socket says 172.21.15.86:46610) 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a1e00700 1 -- 172.21.15.86:0/3920134290 learned_addr learned my addr 172.21.15.86:0/3920134290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.979+0000 7f70a1e00700 1 -- 172.21.15.86:0/3920134290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009550 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.980+0000 7f70a1e00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a419c490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7094009130 tx=0x7f7094011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.980+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7094003ab0 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.980+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7094003c10 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.980+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70940247d0 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.980+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a419cbd0 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.980+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a419d070 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.981+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) v1 ==== 45363+0+0 (secure 0 0 0) 0x7f709401abb0 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.981+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a410c8a0 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.982+0000 7f709aa00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f7088038520 0x7f708803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.982+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f709404c100 con 0x7f70a407ad90 2024-03-22T19:03:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.982+0000 7f709be00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f7088038520 0x7f708803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.982+0000 7f709be00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f7088038520 0x7f708803a9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f708c006fd0 tx=0x7f708c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:13.986+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7094018710 con 0x7f70a407ad90 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.185+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f70a410cac0 con 0x7f70a407ad90 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.186+0000 7f709aa00700 1 -- 172.21.15.86:0/3920134290 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+398 (secure 0 0 0) 0x7f7094018350 con 0x7f70a407ad90 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f7088038520 msgr2=0x7f708803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f7088038520 0x7f708803a9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f708c006fd0 tx=0x7f708c006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 msgr2=0x7f70a419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a419c490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7094009130 tx=0x7f7094011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 shutdown_connections 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f7088038520 0x7f708803a9e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 --2- 172.21.15.86:0/3920134290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a407ad90 0x7f70a419c490 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.188+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 >> 172.21.15.86:0/3920134290 conn(0x7f70a4101a50 msgr2=0x7f70a4102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.189+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 shutdown_connections 2024-03-22T19:03:14.546 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:14.189+0000 7f70a8c00700 1 -- 172.21.15.86:0/3920134290 wait complete. 2024-03-22T19:03:14.547 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-03-22T19:03:15.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:14 smithi086 ceph-mon[38032]: mgrmap e4: x(active, since 2s) 2024-03-22T19:03:15.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3920134290' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T19:03:16.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:16 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/292860976' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-22T19:03:16.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:16 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: ignoring --setuser ceph since I am not root 2024-03-22T19:03:16.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:16 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: ignoring --setgroup ceph since I am not root 2024-03-22T19:03:17.399 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:17 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:17.051+0000 7fa124e26200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T19:03:17.399 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:17 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:17.169+0000 7fa124e26200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T19:03:17.399 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:17 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:17.289+0000 7fa124e26200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T19:03:17.399 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:17 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:17.399+0000 7fa124e26200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T19:03:17.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:17 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/292860976' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-22T19:03:17.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:17 smithi086 ceph-mon[38032]: mgrmap e5: x(active, since 4s) 2024-03-22T19:03:18.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:18 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:18.222+0000 7fa124e26200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:03:18.789 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:18 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:18.536+0000 7fa124e26200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T19:03:18.789 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:18 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:18.649+0000 7fa124e26200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T19:03:19.154 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:18 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:18.788+0000 7fa124e26200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T19:03:19.154 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:18 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:18.984+0000 7fa124e26200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T19:03:19.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:19.154+0000 7fa124e26200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T19:03:19.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:19.267+0000 7fa124e26200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T19:03:20.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:19.735+0000 7fa124e26200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T19:03:20.479 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:20 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:20.222+0000 7fa124e26200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45dce00700 1 Processor -- start 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45dce00700 1 -- start start 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45dce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d80791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45dce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8079780 con 0x7f45d807ad50 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d80791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d80791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46620/0 (socket says 172.21.15.86:46620) 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.849+0000 7f45d5e00700 1 -- 172.21.15.86:0/3090381756 learned_addr learned my addr 172.21.15.86:0/3090381756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.850+0000 7f45d5e00700 1 -- 172.21.15.86:0/3090381756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80798c0 con 0x7f45d807ad50 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.850+0000 7f45d5e00700 1 --2- 172.21.15.86:0/3090381756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d80791b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45c80098a0 tx=0x7f45c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f016d45f192348 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.850+0000 7f45d4a00700 1 -- 172.21.15.86:0/3090381756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c8004160 con 0x7f45d807ad50 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.850+0000 7f45d4a00700 1 -- 172.21.15.86:0/3090381756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f45c80042c0 con 0x7f45d807ad50 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.850+0000 7f45d4a00700 1 -- 172.21.15.86:0/3090381756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c8011a00 con 0x7f45d807ad50 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 -- 172.21.15.86:0/3090381756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 msgr2=0x7f45d80791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 --2- 172.21.15.86:0/3090381756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d80791b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45c80098a0 tx=0x7f45c8009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 -- 172.21.15.86:0/3090381756 shutdown_connections 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 --2- 172.21.15.86:0/3090381756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d80791b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 -- 172.21.15.86:0/3090381756 >> 172.21.15.86:0/3090381756 conn(0x7f45d8101a50 msgr2=0x7f45d8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 -- 172.21.15.86:0/3090381756 shutdown_connections 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.851+0000 7f45dce00700 1 -- 172.21.15.86:0/3090381756 wait complete. 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.852+0000 7f45dce00700 1 Processor -- start 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.852+0000 7f45dce00700 1 -- start start 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.852+0000 7f45dce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.852+0000 7f45dce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d819c990 con 0x7f45d807ad50 2024-03-22T19:03:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.852+0000 7f45d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46632/0 (socket says 172.21.15.86:46632) 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45d5e00700 1 -- 172.21.15.86:0/292860976 learned_addr learned my addr 172.21.15.86:0/292860976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45d5e00700 1 -- 172.21.15.86:0/292860976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8009550 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45d5e00700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d819c450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009130 tx=0x7f45c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c8003b70 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f45c8003cd0 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d819cb90 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c80197d0 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.853+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d819d030 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.854+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) v1 ==== 45363+0+0 (secure 0 0 0) 0x7f45c80199c0 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.855+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d810c8e0 con 0x7f45d807ad50 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.855+0000 7f45cea00700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f45bc0384d0 0x7f45bc03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.855+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f45c804c6e0 con 0x7f45d807ad50 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.855+0000 7f45d5400700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f45bc0384d0 0x7f45bc03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.855+0000 7f45d5400700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f45bc0384d0 0x7f45bc03a990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45c4006fd0 tx=0x7f45c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:15.860+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f45c8018db0 con 0x7f45d807ad50 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.118+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f45d810cad0 con 0x7f45d807ad50 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.433+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 5) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f45c8019c70 con 0x7f45d807ad50 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.435+0000 7f45cea00700 1 -- 172.21.15.86:0/292860976 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f45c8023c00 con 0x7f45d807ad50 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.437+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f45bc0384d0 msgr2=0x7f45bc03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.437+0000 7f45dce00700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f45bc0384d0 0x7f45bc03a990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45c4006fd0 tx=0x7f45c4006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 msgr2=0x7f45d819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d819c450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009130 tx=0x7f45c8011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 shutdown_connections 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f45bc0384d0 0x7f45bc03a990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 --2- 172.21.15.86:0/292860976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d807ad50 0x7f45d819c450 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 >> 172.21.15.86:0/292860976 conn(0x7f45d8101a50 msgr2=0x7f45d8102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 shutdown_connections 2024-03-22T19:03:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:16.438+0000 7f45dce00700 1 -- 172.21.15.86:0/292860976 wait complete. 2024-03-22T19:03:20.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:20 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:20.479+0000 7fa124e26200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-22T19:03:20.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:20 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:20.601+0000 7fa124e26200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T19:03:21.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:21 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2363797660' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.061+0000 7f09a0a00700 1 Processor -- start 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.062+0000 7f09a0a00700 1 -- start start 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.062+0000 7f09a0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c105820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.062+0000 7f09a0a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c105df0 con 0x7f099c105400 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.062+0000 7f0999e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.062+0000 7f0999e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47174/0 (socket says 172.21.15.86:47174) 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.062+0000 7f0999e00700 1 -- 172.21.15.86:0/1423179853 learned_addr learned my addr 172.21.15.86:0/1423179853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.063+0000 7f0999e00700 1 -- 172.21.15.86:0/1423179853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c106600 con 0x7f099c105400 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.063+0000 7f0999e00700 1 --2- 172.21.15.86:0/1423179853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c105820 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f098c0098a0 tx=0x7f098c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b737ea0b92f052f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.063+0000 7f0998a00700 1 -- 172.21.15.86:0/1423179853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f098c004160 con 0x7f099c105400 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.063+0000 7f0998a00700 1 -- 172.21.15.86:0/1423179853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f098c0042c0 con 0x7f099c105400 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.063+0000 7f0998a00700 1 -- 172.21.15.86:0/1423179853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f098c011a00 con 0x7f099c105400 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 -- 172.21.15.86:0/1423179853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 msgr2=0x7f099c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 --2- 172.21.15.86:0/1423179853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c105820 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f098c0098a0 tx=0x7f098c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 -- 172.21.15.86:0/1423179853 shutdown_connections 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 --2- 172.21.15.86:0/1423179853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c105820 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 -- 172.21.15.86:0/1423179853 >> 172.21.15.86:0/1423179853 conn(0x7f099c1009c0 msgr2=0x7f099c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 -- 172.21.15.86:0/1423179853 shutdown_connections 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.064+0000 7f09a0a00700 1 -- 172.21.15.86:0/1423179853 wait complete. 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.066+0000 7f09a0a00700 1 Processor -- start 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.066+0000 7f09a0a00700 1 -- start start 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.066+0000 7f09a0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.066+0000 7f09a0a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c19a900 con 0x7f099c105400 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0999e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0999e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47180/0 (socket says 172.21.15.86:47180) 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0999e00700 1 -- 172.21.15.86:0/2363797660 learned_addr learned my addr 172.21.15.86:0/2363797660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0999e00700 1 -- 172.21.15.86:0/2363797660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009550 con 0x7f099c105400 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0999e00700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c19a3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f098c011710 tx=0x7f098c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f098c003af0 con 0x7f099c105400 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f098c003c50 con 0x7f099c105400 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c19ab00 con 0x7f099c105400 2024-03-22T19:03:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.067+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f098c024780 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.068+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c19afa0 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.068+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f098c024970 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.069+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c1943e0 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.069+0000 7f0992a00700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 0x7f098003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.069+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f098c04bf70 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.069+0000 7f0999400700 1 -- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 msgr2=0x7f098003a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2290148199 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.069+0000 7f0999400700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 0x7f098003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.074+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f098c018b80 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.270+0000 7f0999400700 1 -- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 msgr2=0x7f098003a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2290148199 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.270+0000 7f0999400700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 0x7f098003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.338+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f099c0619c0 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.339+0000 7f0992a00700 1 -- 172.21.15.86:0/2363797660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f098c018350 con 0x7f099c105400 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 msgr2=0x7f098003a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 0x7f098003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 msgr2=0x7f099c19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c19a3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f098c011710 tx=0x7f098c011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 shutdown_connections 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f0980038520 0x7f098003a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 --2- 172.21.15.86:0/2363797660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c105400 0x7f099c19a3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.341+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 >> 172.21.15.86:0/2363797660 conn(0x7f099c1009c0 msgr2=0x7f099c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.342+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 shutdown_connections 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:21.342+0000 7f09a0a00700 1 -- 172.21.15.86:0/2363797660 wait complete. 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-03-22T19:03:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-03-22T19:03:21.912 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:21 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:21.624+0000 7fa124e26200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T19:03:21.912 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:21 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:21.754+0000 7fa124e26200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T19:03:22.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:21 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:21.958+0000 7fa124e26200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T19:03:23.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:22 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:22.777+0000 7fa124e26200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-22T19:03:23.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:23 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:23.016+0000 7fa124e26200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T19:03:23.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:23 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:23.266+0000 7fa124e26200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T19:03:24.751 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:24 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:24.384+0000 7fa124e26200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T19:03:25.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:24 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:24.751+0000 7fa124e26200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:03:25.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:24 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:24.869+0000 7fa124e26200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T19:03:25.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:25.118+0000 7fa124e26200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T19:03:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: Active manager daemon x restarted 2024-03-22T19:03:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: Activating manager daemon x 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: osdmap e2: 0 total, 0 up, 0 in 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: mgrmap e6: x(active, starting, since 0.00404642s) 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: Manager daemon x is now available 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:25 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:26.409 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:26 smithi086 ceph-mon[38032]: Found migration_current of "None". Setting to last migration. 2024-03-22T19:03:26.409 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:26 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-22T19:03:26.409 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:26 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-22T19:03:26.409 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:26 smithi086 ceph-mon[38032]: mgrmap e7: x(active, since 1.00685s) 2024-03-22T19:03:26.565 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.189+0000 7f4637600700 1 Processor -- start 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.189+0000 7f4637600700 1 -- start start 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.190+0000 7f4637600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f4630106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.190+0000 7f4637600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630106ee0 con 0x7f46301064f0 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.190+0000 7f4634c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f4630106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.190+0000 7f4634c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f4630106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47188/0 (socket says 172.21.15.86:47188) 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.190+0000 7f4634c00700 1 -- 172.21.15.86:0/3935347877 learned_addr learned my addr 172.21.15.86:0/3935347877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.190+0000 7f4634c00700 1 -- 172.21.15.86:0/3935347877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46301076f0 con 0x7f46301064f0 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.191+0000 7f4634c00700 1 --2- 172.21.15.86:0/3935347877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f4630106910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46240098a0 tx=0x7f4624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b0021997f76288 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.191+0000 7f462f400700 1 -- 172.21.15.86:0/3935347877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4624004160 con 0x7f46301064f0 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.191+0000 7f462f400700 1 -- 172.21.15.86:0/3935347877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46240042c0 con 0x7f46301064f0 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.191+0000 7f462f400700 1 -- 172.21.15.86:0/3935347877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4624011a00 con 0x7f46301064f0 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 -- 172.21.15.86:0/3935347877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 msgr2=0x7f4630106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 --2- 172.21.15.86:0/3935347877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f4630106910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46240098a0 tx=0x7f4624009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 -- 172.21.15.86:0/3935347877 shutdown_connections 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 --2- 172.21.15.86:0/3935347877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f4630106910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 -- 172.21.15.86:0/3935347877 >> 172.21.15.86:0/3935347877 conn(0x7f4630101a50 msgr2=0x7f4630103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 -- 172.21.15.86:0/3935347877 shutdown_connections 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.192+0000 7f4637600700 1 -- 172.21.15.86:0/3935347877 wait complete. 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.193+0000 7f4637600700 1 Processor -- start 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.193+0000 7f4637600700 1 -- start start 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.194+0000 7f4637600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f463019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.194+0000 7f4637600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019c9d0 con 0x7f46301064f0 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.194+0000 7f4634c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f463019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.194+0000 7f4634c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f463019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47196/0 (socket says 172.21.15.86:47196) 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.194+0000 7f4634c00700 1 -- 172.21.15.86:0/988787026 learned_addr learned my addr 172.21.15.86:0/988787026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.194+0000 7f4634c00700 1 -- 172.21.15.86:0/988787026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624009550 con 0x7f46301064f0 2024-03-22T19:03:26.568 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.195+0000 7f4634c00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f463019c490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4624009130 tx=0x7f4624011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.195+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4624003ab0 con 0x7f46301064f0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.195+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4624003c10 con 0x7f46301064f0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.195+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46240247d0 con 0x7f46301064f0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.195+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463019cbd0 con 0x7f46301064f0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.195+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463019d070 con 0x7f46301064f0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.196+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f462401abb0 con 0x7f46301064f0 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.197+0000 7f462d600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.197+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 --> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f461803b0f0 con 0x7f4618038520 2024-03-22T19:03:26.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.197+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f462404c150 con 0x7f46301064f0 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.197+0000 7f462fe00700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 msgr2=0x7f461803a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2290148199 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.197+0000 7f462fe00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.398+0000 7f462fe00700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 msgr2=0x7f461803a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2290148199 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.398+0000 7f462fe00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.799+0000 7f462fe00700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 msgr2=0x7f461803a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2290148199 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:22.799+0000 7f462fe00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:23.600+0000 7f462fe00700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 msgr2=0x7f461803a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2290148199 2024-03-22T19:03:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:23.600+0000 7f462fe00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:25.123+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f4624021d60 con 0x7f46301064f0 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:25.123+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 msgr2=0x7f461803a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:25.123+0000 7f462d600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.125+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f462402d770 con 0x7f46301064f0 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.125+0000 7f462d600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f461803be00 0x7f461803e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.125+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f461803b0f0 con 0x7f461803be00 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.126+0000 7f462fe00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f461803be00 0x7f461803e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.127+0000 7f462fe00700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f461803be00 0x7f461803e1f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f461c003a10 tx=0x7f461c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.128+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f461803b0f0 con 0x7f461803be00 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.134+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f46300623e0 con 0x7f461803be00 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.134+0000 7f462d600700 1 -- 172.21.15.86:0/988787026 <== mgr.14120 v2:172.21.15.86:6800/2748771516 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f46300623e0 con 0x7f461803be00 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f461803be00 msgr2=0x7f461803e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f461803be00 0x7f461803e1f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f461c003a10 tx=0x7f461c0092b0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 msgr2=0x7f463019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f463019c490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4624009130 tx=0x7f4624011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 shutdown_connections 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f461803be00 0x7f461803e1f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46301064f0 0x7f463019c490 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 --2- 172.21.15.86:0/988787026 >> [v2:172.21.15.86:6800/2290148199,v1:172.21.15.86:6801/2290148199] conn(0x7f4618038520 0x7f461803a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 >> 172.21.15.86:0/988787026 conn(0x7f4630101a50 msgr2=0x7f46301026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 shutdown_connections 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.135+0000 7f4637600700 1 -- 172.21.15.86:0/988787026 wait complete. 2024-03-22T19:03:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-03-22T19:03:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-03-22T19:03:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:27 smithi086 ceph-mon[38032]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T19:03:27.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:27 smithi086 ceph-mon[38032]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T19:03:27.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:27 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:27.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:27 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:27.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:27 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:27.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:27 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.967+0000 7f94d4400700 1 Processor -- start 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.968+0000 7f94d4400700 1 -- start start 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.968+0000 7f94d4400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.968+0000 7f94d4400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc106ee0 con 0x7f94cc1064f0 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.968+0000 7f94d1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.968+0000 7f94d1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47268/0 (socket says 172.21.15.86:47268) 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.968+0000 7f94d1a00700 1 -- 172.21.15.86:0/1539620766 learned_addr learned my addr 172.21.15.86:0/1539620766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.969+0000 7f94d1a00700 1 -- 172.21.15.86:0/1539620766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc1076f0 con 0x7f94cc1064f0 2024-03-22T19:03:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.969+0000 7f94d1a00700 1 --2- 172.21.15.86:0/1539620766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc106910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f94c00098a0 tx=0x7f94c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7221e62456ba6f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.969+0000 7f94cbe00700 1 -- 172.21.15.86:0/1539620766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c0004160 con 0x7f94cc1064f0 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.969+0000 7f94cbe00700 1 -- 172.21.15.86:0/1539620766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94c00042c0 con 0x7f94cc1064f0 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.969+0000 7f94cbe00700 1 -- 172.21.15.86:0/1539620766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c0011a00 con 0x7f94cc1064f0 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 -- 172.21.15.86:0/1539620766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 msgr2=0x7f94cc106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 --2- 172.21.15.86:0/1539620766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc106910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f94c00098a0 tx=0x7f94c0009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 -- 172.21.15.86:0/1539620766 shutdown_connections 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 --2- 172.21.15.86:0/1539620766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc106910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 -- 172.21.15.86:0/1539620766 >> 172.21.15.86:0/1539620766 conn(0x7f94cc101a50 msgr2=0x7f94cc103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 -- 172.21.15.86:0/1539620766 shutdown_connections 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.970+0000 7f94d4400700 1 -- 172.21.15.86:0/1539620766 wait complete. 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d4400700 1 Processor -- start 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d4400700 1 -- start start 2024-03-22T19:03:27.570 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d4400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d4400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc19a7d0 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47276/0 (socket says 172.21.15.86:47276) 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.971+0000 7f94d1a00700 1 -- 172.21.15.86:0/1470549336 learned_addr learned my addr 172.21.15.86:0/1470549336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94d1a00700 1 -- 172.21.15.86:0/1470549336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0009550 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94d1a00700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc19a290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009130 tx=0x7f94c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c0003ab0 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94c0003c10 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc19a9d0 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c00247d0 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.972+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc19ae70 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.973+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 7) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f94c001abb0 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.973+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc10a920 con 0x7f94cc1064f0 2024-03-22T19:03:27.571 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.974+0000 7f94ca000700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f94b40383a0 0x7f94b403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.974+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f94c004c3c0 con 0x7f94cc1064f0 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.974+0000 7f94d1000700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f94b40383a0 0x7f94b403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.974+0000 7f94d1000700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f94b40383a0 0x7f94b403a860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f94b8006fd0 tx=0x7f94b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:26.979+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f94c0018920 con 0x7f94cc1064f0 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.196+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f94cc102dd0 con 0x7f94b40383a0 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.205+0000 7f94ca000700 1 -- 172.21.15.86:0/1470549336 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f94cc102dd0 con 0x7f94b40383a0 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f94b40383a0 msgr2=0x7f94b403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f94b40383a0 0x7f94b403a860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f94b8006fd0 tx=0x7f94b8006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 msgr2=0x7f94cc19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc19a290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009130 tx=0x7f94c0011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 shutdown_connections 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f94b40383a0 0x7f94b403a860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 --2- 172.21.15.86:0/1470549336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc1064f0 0x7f94cc19a290 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:27.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 >> 172.21.15.86:0/1470549336 conn(0x7f94cc101a50 msgr2=0x7f94cc1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:27.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 shutdown_connections 2024-03-22T19:03:27.573 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:27.208+0000 7f94d4400700 1 -- 172.21.15.86:0/1470549336 wait complete. 2024-03-22T19:03:28.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-22T19:03:28.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.024+0000 7f08ed400700 1 Processor -- start 2024-03-22T19:03:28.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.024+0000 7f08ed400700 1 -- start start 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.024+0000 7f08ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e8106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.024+0000 7f08ed400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8106ee0 con 0x7f08e81064f0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e8106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e8106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47290/0 (socket says 172.21.15.86:47290) 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e6800700 1 -- 172.21.15.86:0/2750047887 learned_addr learned my addr 172.21.15.86:0/2750047887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e6800700 1 -- 172.21.15.86:0/2750047887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e81076f0 con 0x7f08e81064f0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e6800700 1 --2- 172.21.15.86:0/2750047887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e8106910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08d80098a0 tx=0x7f08d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b319608641bd7f48 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e5400700 1 -- 172.21.15.86:0/2750047887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08d8004160 con 0x7f08e81064f0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e5400700 1 -- 172.21.15.86:0/2750047887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f08d80042c0 con 0x7f08e81064f0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.025+0000 7f08e5400700 1 -- 172.21.15.86:0/2750047887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08d8011a00 con 0x7f08e81064f0 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 -- 172.21.15.86:0/2750047887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 msgr2=0x7f08e8106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 --2- 172.21.15.86:0/2750047887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e8106910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08d80098a0 tx=0x7f08d8009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 -- 172.21.15.86:0/2750047887 shutdown_connections 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 --2- 172.21.15.86:0/2750047887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e8106910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 -- 172.21.15.86:0/2750047887 >> 172.21.15.86:0/2750047887 conn(0x7f08e8101a50 msgr2=0x7f08e8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 -- 172.21.15.86:0/2750047887 shutdown_connections 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.026+0000 7f08ed400700 1 -- 172.21.15.86:0/2750047887 wait complete. 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.027+0000 7f08ed400700 1 Processor -- start 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.027+0000 7f08ed400700 1 -- start start 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08ed400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e819ca20 con 0x7f08e81064f0 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08e6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08e6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47304/0 (socket says 172.21.15.86:47304) 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08e6800700 1 -- 172.21.15.86:0/2999490885 learned_addr learned my addr 172.21.15.86:0/2999490885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08e6800700 1 -- 172.21.15.86:0/2999490885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8009550 con 0x7f08e81064f0 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.028+0000 7f08e6800700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e819c4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009130 tx=0x7f08d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.029+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08d8003ab0 con 0x7f08e81064f0 2024-03-22T19:03:28.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.029+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f08d8003c10 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.029+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08d80247d0 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.029+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e819cc20 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.029+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e819d0c0 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.030+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f08d801abb0 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.030+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e810a920 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.030+0000 7f08df400700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f08cc0384d0 0x7f08cc03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.030+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f08d804c5f0 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.031+0000 7f08e5e00700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f08cc0384d0 0x7f08cc03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.031+0000 7f08e5e00700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f08cc0384d0 0x7f08cc03a990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08d4006fd0 tx=0x7f08d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.035+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08d8018ba0 con 0x7f08e81064f0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.237+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f08e8102d10 con 0x7f08cc0384d0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.238+0000 7f08df400700 1 -- 172.21.15.86:0/2999490885 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f08e8102d10 con 0x7f08cc0384d0 2024-03-22T19:03:28.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f08cc0384d0 msgr2=0x7f08cc03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f08cc0384d0 0x7f08cc03a990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08d4006fd0 tx=0x7f08d4006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 msgr2=0x7f08e819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e819c4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009130 tx=0x7f08d8011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 shutdown_connections 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f08cc0384d0 0x7f08cc03a990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 --2- 172.21.15.86:0/2999490885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e81064f0 0x7f08e819c4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 >> 172.21.15.86:0/2999490885 conn(0x7f08e8101a50 msgr2=0x7f08e8102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.240+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 shutdown_connections 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.241+0000 7f08ed400700 1 -- 172.21.15.86:0/2999490885 wait complete. 2024-03-22T19:03:28.627 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-03-22T19:03:28.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:26] ENGINE Bus STARTING 2024-03-22T19:03:28.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:26] ENGINE Serving on http://172.21.15.86:8765 2024-03-22T19:03:28.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:26] ENGINE Serving on https://172.21.15.86:7150 2024-03-22T19:03:28.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:26] ENGINE Bus STARTED 2024-03-22T19:03:28.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:28.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:28.628 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:28 smithi086 ceph-mon[38032]: mgrmap e8: x(active, since 2s) 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-mon[38032]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: Generating public/private rsa key pair. 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: Your identification has been saved in /tmp/tmppkka34nd/key. 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: Your public key has been saved in /tmp/tmppkka34nd/key.pub. 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: The key fingerprint is: 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: SHA256:GFOIQYN0cM2cm7ppAerA/43RbzXD5oBCW2xzNl7bhR8 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: The key's randomart image is: 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: +---[RSA 3072]----+ 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: | .o+** o. | 2024-03-22T19:03:29.539 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: | .o..*. | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: | +o . | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: | . .oO + . . E | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: |.. ...= S + o o .| 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: |o. oo.. o B . . | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: |o . =.. = o | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: | . .+ + .. . | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: | ..o ... | 2024-03-22T19:03:29.540 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:29 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: +----[SHA256]-----+ 2024-03-22T19:03:29.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.997+0000 7f2cd1c00700 1 Processor -- start 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.998+0000 7f2cd1c00700 1 -- start start 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.998+0000 7f2cd1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc0791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.998+0000 7f2cd1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0797a0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.998+0000 7f2ccaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc0791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.998+0000 7f2ccaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc0791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49480/0 (socket says 172.21.15.86:49480) 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.998+0000 7f2ccaa00700 1 -- 172.21.15.86:0/3413355066 learned_addr learned my addr 172.21.15.86:0/3413355066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.999+0000 7f2ccaa00700 1 -- 172.21.15.86:0/3413355066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0798e0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.999+0000 7f2ccaa00700 1 --2- 172.21.15.86:0/3413355066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc0791d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0098a0 tx=0x7f2cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc0294162a9c9d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.999+0000 7f2cca000700 1 -- 172.21.15.86:0/3413355066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc004160 con 0x7f2ccc07ad70 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.999+0000 7f2cca000700 1 -- 172.21.15.86:0/3413355066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cbc0042c0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:28.999+0000 7f2cca000700 1 -- 172.21.15.86:0/3413355066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc011a00 con 0x7f2ccc07ad70 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 -- 172.21.15.86:0/3413355066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 msgr2=0x7f2ccc0791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:29.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 --2- 172.21.15.86:0/3413355066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc0791d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0098a0 tx=0x7f2cbc009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 -- 172.21.15.86:0/3413355066 shutdown_connections 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 --2- 172.21.15.86:0/3413355066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc0791d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 -- 172.21.15.86:0/3413355066 >> 172.21.15.86:0/3413355066 conn(0x7f2ccc101a50 msgr2=0x7f2ccc103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 -- 172.21.15.86:0/3413355066 shutdown_connections 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.000+0000 7f2cd1c00700 1 -- 172.21.15.86:0/3413355066 wait complete. 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.001+0000 7f2cd1c00700 1 Processor -- start 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.001+0000 7f2cd1c00700 1 -- start start 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.001+0000 7f2cd1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.001+0000 7f2cd1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc19c9d0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.001+0000 7f2ccaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2ccaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49484/0 (socket says 172.21.15.86:49484) 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2ccaa00700 1 -- 172.21.15.86:0/309587561 learned_addr learned my addr 172.21.15.86:0/309587561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2ccaa00700 1 -- 172.21.15.86:0/309587561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc009550 con 0x7f2ccc07ad70 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2ccaa00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc19c490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009130 tx=0x7f2cbc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc003ab0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cbc003c10 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc19cbd0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc0247d0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.002+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc19d070 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.003+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f2cbc01a430 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.004+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc10c8c0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.004+0000 7f2cc3400700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f2cb0038520 0x7f2cb003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.004+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2cbc04c5c0 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.004+0000 7f2cc3e00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f2cb0038520 0x7f2cb003a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.005+0000 7f2cc3e00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f2cb0038520 0x7f2cb003a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4006fd0 tx=0x7f2cb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.009+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2cbc018b40 con 0x7f2ccc07ad70 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.213+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ccc102dd0 con 0x7f2cb0038520 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.324+0000 7f2cc3400700 1 -- 172.21.15.86:0/309587561 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2ccc102dd0 con 0x7f2cb0038520 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.326+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f2cb0038520 msgr2=0x7f2cb003a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:29.700 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.326+0000 7f2cd1c00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f2cb0038520 0x7f2cb003a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4006fd0 tx=0x7f2cb4006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.326+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 msgr2=0x7f2ccc19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.326+0000 7f2cd1c00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc19c490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009130 tx=0x7f2cbc011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.327+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 shutdown_connections 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.327+0000 7f2cd1c00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f2cb0038520 0x7f2cb003a9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.327+0000 7f2cd1c00700 1 --2- 172.21.15.86:0/309587561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc07ad70 0x7f2ccc19c490 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.327+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 >> 172.21.15.86:0/309587561 conn(0x7f2ccc101a50 msgr2=0x7f2ccc1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.327+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 shutdown_connections 2024-03-22T19:03:29.701 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:29.327+0000 7f2cd1c00700 1 -- 172.21.15.86:0/309587561 wait complete. 2024-03-22T19:03:30.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:30 smithi086 ceph-mon[38032]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:30.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:30 smithi086 ceph-mon[38032]: Generating ssh key... 2024-03-22T19:03:30.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:30 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:30.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:30 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:30.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.138+0000 7fa358400700 1 Processor -- start 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.138+0000 7fa358400700 1 -- start start 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.139+0000 7fa358400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa350106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.139+0000 7fa358400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350106ee0 con 0x7fa3501064f0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.139+0000 7fa355a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa350106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.139+0000 7fa355a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa350106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49492/0 (socket says 172.21.15.86:49492) 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.139+0000 7fa355a00700 1 -- 172.21.15.86:0/3177690464 learned_addr learned my addr 172.21.15.86:0/3177690464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.139+0000 7fa355a00700 1 -- 172.21.15.86:0/3177690464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3501076f0 con 0x7fa3501064f0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.140+0000 7fa355a00700 1 --2- 172.21.15.86:0/3177690464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa350106910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098a0 tx=0x7fa344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ae012a3ae778d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.140+0000 7fa34fe00700 1 -- 172.21.15.86:0/3177690464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa344004160 con 0x7fa3501064f0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.140+0000 7fa34fe00700 1 -- 172.21.15.86:0/3177690464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3440042c0 con 0x7fa3501064f0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.140+0000 7fa34fe00700 1 -- 172.21.15.86:0/3177690464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa344011a00 con 0x7fa3501064f0 2024-03-22T19:03:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.140+0000 7fa358400700 1 -- 172.21.15.86:0/3177690464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 msgr2=0x7fa350106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.140+0000 7fa358400700 1 --2- 172.21.15.86:0/3177690464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa350106910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098a0 tx=0x7fa344009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.141+0000 7fa358400700 1 -- 172.21.15.86:0/3177690464 shutdown_connections 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.141+0000 7fa358400700 1 --2- 172.21.15.86:0/3177690464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa350106910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.141+0000 7fa358400700 1 -- 172.21.15.86:0/3177690464 >> 172.21.15.86:0/3177690464 conn(0x7fa350101a50 msgr2=0x7fa350103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.141+0000 7fa358400700 1 -- 172.21.15.86:0/3177690464 shutdown_connections 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.141+0000 7fa358400700 1 -- 172.21.15.86:0/3177690464 wait complete. 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.142+0000 7fa358400700 1 Processor -- start 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.142+0000 7fa358400700 1 -- start start 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.142+0000 7fa358400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa35007a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.142+0000 7fa358400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35007af30 con 0x7fa3501064f0 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.142+0000 7fa355a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa35007a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa355a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa35007a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49494/0 (socket says 172.21.15.86:49494) 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa355a00700 1 -- 172.21.15.86:0/1135163725 learned_addr learned my addr 172.21.15.86:0/1135163725 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa355a00700 1 -- 172.21.15.86:0/1135163725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344009550 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa355a00700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa35007a9f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa344009130 tx=0x7fa344011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa344003ab0 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa344003c10 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa350079170 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3440247d0 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.143+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa350079610 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.144+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7fa34401abb0 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.145+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3500623e0 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.145+0000 7fa34e000700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fa338038520 0x7fa33803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.145+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa34404c580 con 0x7fa3501064f0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.145+0000 7fa355000700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fa338038520 0x7fa33803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.146+0000 7fa355000700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fa338038520 0x7fa33803a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006fd0 tx=0x7fa33c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.150+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa344018920 con 0x7fa3501064f0 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.354+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa35007a070 con 0x7fa338038520 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.355+0000 7fa34e000700 1 -- 172.21.15.86:0/1135163725 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa35007a070 con 0x7fa338038520 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fa338038520 msgr2=0x7fa33803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fa338038520 0x7fa33803a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006fd0 tx=0x7fa33c006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 msgr2=0x7fa35007a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa35007a9f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa344009130 tx=0x7fa344011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 shutdown_connections 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fa338038520 0x7fa33803a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 --2- 172.21.15.86:0/1135163725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3501064f0 0x7fa35007a9f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 >> 172.21.15.86:0/1135163725 conn(0x7fa350101a50 msgr2=0x7fa3501026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 shutdown_connections 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:30.357+0000 7fa358400700 1 -- 172.21.15.86:0/1135163725 wait complete. 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-22T19:03:30.976 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-03-22T19:03:30.977 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-03-22T19:03:31.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:31 smithi086 ceph-mon[38032]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:32.835 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:32 smithi086 ceph-mon[38032]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:32.835 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:32 smithi086 ceph-mon[38032]: Deploying cephadm binary to smithi086 2024-03-22T19:03:34.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.360+0000 7fc5cbc00700 1 Processor -- start 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.361+0000 7fc5cbc00700 1 -- start start 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.361+0000 7fc5cbc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c40791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.361+0000 7fc5cbc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4079780 con 0x7fc5c407ad50 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.361+0000 7fc5c9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c40791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.361+0000 7fc5c9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c40791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49496/0 (socket says 172.21.15.86:49496) 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.361+0000 7fc5c9200700 1 -- 172.21.15.86:0/1332679896 learned_addr learned my addr 172.21.15.86:0/1332679896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.362+0000 7fc5c9200700 1 -- 172.21.15.86:0/1332679896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40798c0 con 0x7fc5c407ad50 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.362+0000 7fc5c9200700 1 --2- 172.21.15.86:0/1332679896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c40791b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80098a0 tx=0x7fc5b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af66047de7d0a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.362+0000 7fc5c3400700 1 -- 172.21.15.86:0/1332679896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b8004160 con 0x7fc5c407ad50 2024-03-22T19:03:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.362+0000 7fc5c3400700 1 -- 172.21.15.86:0/1332679896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5b80042c0 con 0x7fc5c407ad50 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.362+0000 7fc5c3400700 1 -- 172.21.15.86:0/1332679896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b8011a00 con 0x7fc5c407ad50 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 -- 172.21.15.86:0/1332679896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 msgr2=0x7fc5c40791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 --2- 172.21.15.86:0/1332679896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c40791b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80098a0 tx=0x7fc5b8009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 -- 172.21.15.86:0/1332679896 shutdown_connections 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 --2- 172.21.15.86:0/1332679896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c40791b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 -- 172.21.15.86:0/1332679896 >> 172.21.15.86:0/1332679896 conn(0x7fc5c4101a50 msgr2=0x7fc5c4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 -- 172.21.15.86:0/1332679896 shutdown_connections 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.363+0000 7fc5cbc00700 1 -- 172.21.15.86:0/1332679896 wait complete. 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.364+0000 7fc5cbc00700 1 Processor -- start 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.364+0000 7fc5cbc00700 1 -- start start 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.364+0000 7fc5cbc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c4109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.364+0000 7fc5cbc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4109c90 con 0x7fc5c407ad50 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c4109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:34.933 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c4109750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49498/0 (socket says 172.21.15.86:49498) 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c9200700 1 -- 172.21.15.86:0/3578333621 learned_addr learned my addr 172.21.15.86:0/3578333621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c9200700 1 -- 172.21.15.86:0/3578333621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b8009550 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c9200700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c4109750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009130 tx=0x7fc5b8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b8003b70 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5b8003cd0 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b80197d0 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.365+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c4109e90 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.366+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c41062e0 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.367+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7fc5b801abb0 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.367+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c410c8e0 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.367+0000 7fc5c1600700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fc5ac0384d0 0x7fc5ac03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.367+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc5b804c6d0 con 0x7fc5c407ad50 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.367+0000 7fc5c3e00700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fc5ac0384d0 0x7fc5ac03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:34.934 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.368+0000 7fc5c3e00700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fc5ac0384d0 0x7fc5ac03a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0006fd0 tx=0x7fc5b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.372+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5b801a430 con 0x7fc5c407ad50 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:31.579+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5c4106f90 con 0x7fc5ac0384d0 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.543+0000 7fc5c1600700 1 -- 172.21.15.86:0/3578333621 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fc5c4106f90 con 0x7fc5ac0384d0 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.545+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fc5ac0384d0 msgr2=0x7fc5ac03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.545+0000 7fc5cbc00700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fc5ac0384d0 0x7fc5ac03a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0006fd0 tx=0x7fc5b0006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.545+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 msgr2=0x7fc5c4109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.545+0000 7fc5cbc00700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c4109750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009130 tx=0x7fc5b8011ad0 comp rx=0 tx=0).stop 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.546+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 shutdown_connections 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.546+0000 7fc5cbc00700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7fc5ac0384d0 0x7fc5ac03a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.546+0000 7fc5cbc00700 1 --2- 172.21.15.86:0/3578333621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c407ad50 0x7fc5c4109750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.546+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 >> 172.21.15.86:0/3578333621 conn(0x7fc5c4101a50 msgr2=0x7fc5c4102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.546+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 shutdown_connections 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:34.546+0000 7fc5cbc00700 1 -- 172.21.15.86:0/3578333621 wait complete. 2024-03-22T19:03:34.935 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-03-22T19:03:35.592 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:35 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:35.592 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:35 smithi086 ceph-mon[38032]: Added host smithi086 2024-03-22T19:03:35.592 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:35 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.484+0000 7f7a62600700 1 Processor -- start 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.485+0000 7f7a62600700 1 -- start start 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.485+0000 7f7a62600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.485+0000 7f7a62600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c079780 con 0x7f7a5c07ad50 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.486+0000 7f7a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:36.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.486+0000 7f7a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c0791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49502/0 (socket says 172.21.15.86:49502) 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.486+0000 7f7a5b400700 1 -- 172.21.15.86:0/177225401 learned_addr learned my addr 172.21.15.86:0/177225401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.486+0000 7f7a5b400700 1 -- 172.21.15.86:0/177225401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0798c0 con 0x7f7a5c07ad50 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.486+0000 7f7a5b400700 1 --2- 172.21.15.86:0/177225401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c0791b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098a0 tx=0x7f7a4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c680ce854af0316 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a5a000700 1 -- 172.21.15.86:0/177225401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a4c004160 con 0x7f7a5c07ad50 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a5a000700 1 -- 172.21.15.86:0/177225401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a4c0042c0 con 0x7f7a5c07ad50 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a5a000700 1 -- 172.21.15.86:0/177225401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a4c011a00 con 0x7f7a5c07ad50 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a62600700 1 -- 172.21.15.86:0/177225401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 msgr2=0x7f7a5c0791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a62600700 1 --2- 172.21.15.86:0/177225401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c0791b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098a0 tx=0x7f7a4c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:36.300 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a62600700 1 -- 172.21.15.86:0/177225401 shutdown_connections 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a62600700 1 --2- 172.21.15.86:0/177225401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c0791b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.487+0000 7f7a62600700 1 -- 172.21.15.86:0/177225401 >> 172.21.15.86:0/177225401 conn(0x7f7a5c101ab0 msgr2=0x7f7a5c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.488+0000 7f7a62600700 1 -- 172.21.15.86:0/177225401 shutdown_connections 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.488+0000 7f7a62600700 1 -- 172.21.15.86:0/177225401 wait complete. 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a62600700 1 Processor -- start 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a62600700 1 -- start start 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a62600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a62600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c19ca00 con 0x7f7a5c07ad50 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49504/0 (socket says 172.21.15.86:49504) 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.489+0000 7f7a5b400700 1 -- 172.21.15.86:0/1530164395 learned_addr learned my addr 172.21.15.86:0/1530164395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a5b400700 1 -- 172.21.15.86:0/1530164395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009550 con 0x7f7a5c07ad50 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a5b400700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c19c4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c011a20 tx=0x7f7a4c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a4c003af0 con 0x7f7a5c07ad50 2024-03-22T19:03:36.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a4c003c50 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a4c019760 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c19cc00 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.490+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c19d0a0 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.491+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f7a4c01ab20 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.492+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c10c900 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.492+0000 7f7a53e00700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7a400384d0 0x7f7a4003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.492+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a4c04c640 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.492+0000 7f7a5aa00700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7a400384d0 0x7f7a4003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.493+0000 7f7a5aa00700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7a400384d0 0x7f7a4003a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006fd0 tx=0x7f7a48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.502+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a4c01a7b0 con 0x7f7a5c07ad50 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.728+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7a5c102da0 con 0x7f7a400384d0 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.734+0000 7f7a53e00700 1 -- 172.21.15.86:0/1530164395 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7a5c102da0 con 0x7f7a400384d0 2024-03-22T19:03:36.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.736+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7a400384d0 msgr2=0x7f7a4003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.736+0000 7f7a62600700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7a400384d0 0x7f7a4003a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006fd0 tx=0x7f7a48006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.736+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 msgr2=0x7f7a5c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.736+0000 7f7a62600700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c19c4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c011a20 tx=0x7f7a4c011b00 comp rx=0 tx=0).stop 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.737+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 shutdown_connections 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.737+0000 7f7a62600700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7a400384d0 0x7f7a4003a990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.737+0000 7f7a62600700 1 --2- 172.21.15.86:0/1530164395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c07ad50 0x7f7a5c19c4c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.737+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 >> 172.21.15.86:0/1530164395 conn(0x7f7a5c101ab0 msgr2=0x7f7a5c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.737+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 shutdown_connections 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:35.737+0000 7f7a62600700 1 -- 172.21.15.86:0/1530164395 wait complete. 2024-03-22T19:03:36.303 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-03-22T19:03:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:36 smithi086 ceph-mon[38032]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:36 smithi086 ceph-mon[38032]: Saving service mon spec with placement count:5 2024-03-22T19:03:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:36 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:36 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:36 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.799+0000 7f3c87600700 1 Processor -- start 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.799+0000 7f3c87600700 1 -- start start 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c87600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c800791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c87600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800797a0 con 0x7f3c8007ad70 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c84c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c800791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:37.491 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c84c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c800791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49510/0 (socket says 172.21.15.86:49510) 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c84c00700 1 -- 172.21.15.86:0/3966580045 learned_addr learned my addr 172.21.15.86:0/3966580045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c84c00700 1 -- 172.21.15.86:0/3966580045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800798e0 con 0x7f3c8007ad70 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.800+0000 7f3c84c00700 1 --2- 172.21.15.86:0/3966580045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c800791d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c740098a0 tx=0x7f3c74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b614a4373202a681 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.801+0000 7f3c7f400700 1 -- 172.21.15.86:0/3966580045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c74004160 con 0x7f3c8007ad70 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.801+0000 7f3c7f400700 1 -- 172.21.15.86:0/3966580045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c740042c0 con 0x7f3c8007ad70 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.801+0000 7f3c7f400700 1 -- 172.21.15.86:0/3966580045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c740119b0 con 0x7f3c8007ad70 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.801+0000 7f3c87600700 1 -- 172.21.15.86:0/3966580045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 msgr2=0x7f3c800791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:37.492 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.801+0000 7f3c87600700 1 --2- 172.21.15.86:0/3966580045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c800791d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c740098a0 tx=0x7f3c74009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.802+0000 7f3c87600700 1 -- 172.21.15.86:0/3966580045 shutdown_connections 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.802+0000 7f3c87600700 1 --2- 172.21.15.86:0/3966580045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c800791d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.802+0000 7f3c87600700 1 -- 172.21.15.86:0/3966580045 >> 172.21.15.86:0/3966580045 conn(0x7f3c80101a50 msgr2=0x7f3c80103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.802+0000 7f3c87600700 1 -- 172.21.15.86:0/3966580045 shutdown_connections 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.802+0000 7f3c87600700 1 -- 172.21.15.86:0/3966580045 wait complete. 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c87600700 1 Processor -- start 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c87600700 1 -- start start 2024-03-22T19:03:37.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c87600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c8019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c87600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8019c9f0 con 0x7f3c8007ad70 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c84c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c8019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c84c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c8019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49526/0 (socket says 172.21.15.86:49526) 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.803+0000 7f3c84c00700 1 -- 172.21.15.86:0/770004274 learned_addr learned my addr 172.21.15.86:0/770004274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c84c00700 1 -- 172.21.15.86:0/770004274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74009550 con 0x7f3c8007ad70 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c84c00700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c8019c4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c74003e60 tx=0x7f3c74011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:37.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c74003a60 con 0x7f3c8007ad70 2024-03-22T19:03:37.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c74003bc0 con 0x7f3c8007ad70 2024-03-22T19:03:37.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c740196c0 con 0x7f3c8007ad70 2024-03-22T19:03:37.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8019cbf0 con 0x7f3c8007ad70 2024-03-22T19:03:37.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.804+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8019d090 con 0x7f3c8007ad70 2024-03-22T19:03:37.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.805+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f3c7401a430 con 0x7f3c8007ad70 2024-03-22T19:03:37.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.806+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8010c910 con 0x7f3c8007ad70 2024-03-22T19:03:37.497 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.806+0000 7f3c7d600700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f3c680384b0 0x7f3c6803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.806+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3c7404c750 con 0x7f3c8007ad70 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.806+0000 7f3c7fe00700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f3c680384b0 0x7f3c6803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.807+0000 7f3c7fe00700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f3c680384b0 0x7f3c6803a970 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c006fd0 tx=0x7f3c6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:36.811+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3c74010560 con 0x7f3c8007ad70 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.040+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3c8019fd40 con 0x7f3c680384b0 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.048+0000 7f3c7d600700 1 -- 172.21.15.86:0/770004274 <== mgr.14120 v2:172.21.15.86:6800/2748771516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3c8019fd40 con 0x7f3c680384b0 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f3c680384b0 msgr2=0x7f3c6803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f3c680384b0 0x7f3c6803a970 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c006fd0 tx=0x7f3c6c006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 msgr2=0x7f3c8019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c8019c4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c74003e60 tx=0x7f3c74011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 shutdown_connections 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f3c680384b0 0x7f3c6803a970 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:37.498 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 --2- 172.21.15.86:0/770004274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c8007ad70 0x7f3c8019c4b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:37.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 >> 172.21.15.86:0/770004274 conn(0x7f3c80101a50 msgr2=0x7f3c801026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:37.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 shutdown_connections 2024-03-22T19:03:37.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:37.050+0000 7f3c87600700 1 -- 172.21.15.86:0/770004274 wait complete. 2024-03-22T19:03:38.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:38 smithi086 ceph-mon[38032]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:38.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:38 smithi086 ceph-mon[38032]: Saving service mgr spec with placement count:2 2024-03-22T19:03:38.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:38 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.235+0000 7f635f200700 1 Processor -- start 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.235+0000 7f635f200700 1 -- start start 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.235+0000 7f635f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f63580791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.236+0000 7f635f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580797a0 con 0x7f635807ad70 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.236+0000 7f635de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f63580791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.236+0000 7f635de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f63580791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49528/0 (socket says 172.21.15.86:49528) 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.236+0000 7f635de00700 1 -- 172.21.15.86:0/2151895038 learned_addr learned my addr 172.21.15.86:0/2151895038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.236+0000 7f635de00700 1 -- 172.21.15.86:0/2151895038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63580798e0 con 0x7f635807ad70 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.237+0000 7f635de00700 1 --2- 172.21.15.86:0/2151895038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f63580791d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f634c0098a0 tx=0x7f634c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca88d3fc9036335 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.237+0000 7f635ca00700 1 -- 172.21.15.86:0/2151895038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f634c004160 con 0x7f635807ad70 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.237+0000 7f635ca00700 1 -- 172.21.15.86:0/2151895038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634c0042c0 con 0x7f635807ad70 2024-03-22T19:03:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.237+0000 7f635ca00700 1 -- 172.21.15.86:0/2151895038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f634c0119b0 con 0x7f635807ad70 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 -- 172.21.15.86:0/2151895038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 msgr2=0x7f63580791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 --2- 172.21.15.86:0/2151895038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f63580791d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f634c0098a0 tx=0x7f634c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 -- 172.21.15.86:0/2151895038 shutdown_connections 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 --2- 172.21.15.86:0/2151895038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f63580791d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 -- 172.21.15.86:0/2151895038 >> 172.21.15.86:0/2151895038 conn(0x7f63581018c0 msgr2=0x7f6358103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 -- 172.21.15.86:0/2151895038 shutdown_connections 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.238+0000 7f635f200700 1 -- 172.21.15.86:0/2151895038 wait complete. 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.239+0000 7f635f200700 1 Processor -- start 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.239+0000 7f635f200700 1 -- start start 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f635819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635819c7f0 con 0x7f635807ad70 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f635819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f635819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49538/0 (socket says 172.21.15.86:49538) 2024-03-22T19:03:38.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635de00700 1 -- 172.21.15.86:0/4290485925 learned_addr learned my addr 172.21.15.86:0/4290485925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635de00700 1 -- 172.21.15.86:0/4290485925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c009550 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.240+0000 7f635de00700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f635819c2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f634c003e60 tx=0x7f634c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.241+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f634c003a60 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.241+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634c003bc0 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.241+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635819c9f0 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.241+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f634c0196c0 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.241+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635819ce90 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.242+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f634c019930 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.242+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635810c740 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.242+0000 7f6352a00700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f6340038500 0x7f634003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.242+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f634c04c6e0 con 0x7f635807ad70 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.243+0000 7f635d400700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f6340038500 0x7f634003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.243+0000 7f635d400700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f6340038500 0x7f634003a9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6348006fd0 tx=0x7f6348006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:38.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.247+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f634c010bb0 con 0x7f635807ad70 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.458+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f635819fb40 con 0x7f635807ad70 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.465+0000 7f6352a00700 1 -- 172.21.15.86:0/4290485925 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f634c010380 con 0x7f635807ad70 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.467+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f6340038500 msgr2=0x7f634003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.467+0000 7f635f200700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f6340038500 0x7f634003a9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6348006fd0 tx=0x7f6348006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.467+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 msgr2=0x7f635819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.467+0000 7f635f200700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f635819c2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f634c003e60 tx=0x7f634c011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.468+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 shutdown_connections 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.468+0000 7f635f200700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f6340038500 0x7f634003a9c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.468+0000 7f635f200700 1 --2- 172.21.15.86:0/4290485925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f635807ad70 0x7f635819c2b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.468+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 >> 172.21.15.86:0/4290485925 conn(0x7f63581018c0 msgr2=0x7f6358102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.468+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 shutdown_connections 2024-03-22T19:03:38.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:38.468+0000 7f635f200700 1 -- 172.21.15.86:0/4290485925 wait complete. 2024-03-22T19:03:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:39 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:39 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/4290485925' entity='client.admin' 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.283+0000 7f42bd200700 1 Processor -- start 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.283+0000 7f42bd200700 1 -- start start 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.284+0000 7f42bd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b8079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.284+0000 7f42bd200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80797e0 con 0x7f42b807adb0 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.284+0000 7f42b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b8079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.284+0000 7f42b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b8079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41900/0 (socket says 172.21.15.86:41900) 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.284+0000 7f42b5e00700 1 -- 172.21.15.86:0/1294245017 learned_addr learned my addr 172.21.15.86:0/1294245017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.284+0000 7f42b5e00700 1 -- 172.21.15.86:0/1294245017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b8079920 con 0x7f42b807adb0 2024-03-22T19:03:39.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.285+0000 7f42b5e00700 1 --2- 172.21.15.86:0/1294245017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b8079210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42a80098a0 tx=0x7f42a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29896c945449ccaf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.285+0000 7f42b4a00700 1 -- 172.21.15.86:0/1294245017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a8004160 con 0x7f42b807adb0 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.285+0000 7f42b4a00700 1 -- 172.21.15.86:0/1294245017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42a80042c0 con 0x7f42b807adb0 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.285+0000 7f42b4a00700 1 -- 172.21.15.86:0/1294245017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a80119b0 con 0x7f42b807adb0 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.285+0000 7f42bd200700 1 -- 172.21.15.86:0/1294245017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 msgr2=0x7f42b8079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.285+0000 7f42bd200700 1 --2- 172.21.15.86:0/1294245017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b8079210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42a80098a0 tx=0x7f42a8009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.286+0000 7f42bd200700 1 -- 172.21.15.86:0/1294245017 shutdown_connections 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.286+0000 7f42bd200700 1 --2- 172.21.15.86:0/1294245017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b8079210 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.286+0000 7f42bd200700 1 -- 172.21.15.86:0/1294245017 >> 172.21.15.86:0/1294245017 conn(0x7f42b8101ab0 msgr2=0x7f42b8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.286+0000 7f42bd200700 1 -- 172.21.15.86:0/1294245017 shutdown_connections 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.286+0000 7f42bd200700 1 -- 172.21.15.86:0/1294245017 wait complete. 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42bd200700 1 Processor -- start 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42bd200700 1 -- start start 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42bd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b81a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:39.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42bd200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81a0e70 con 0x7f42b807adb0 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b81a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b81a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41904/0 (socket says 172.21.15.86:41904) 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.287+0000 7f42b5e00700 1 -- 172.21.15.86:0/3202170317 learned_addr learned my addr 172.21.15.86:0/3202170317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.288+0000 7f42b5e00700 1 -- 172.21.15.86:0/3202170317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8009550 con 0x7f42b807adb0 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.288+0000 7f42b5e00700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b81a0930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42a8011670 tx=0x7f42a8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.288+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a8003af0 con 0x7f42b807adb0 2024-03-22T19:03:39.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.288+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42a8003c50 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.288+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b81a1070 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.288+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a8024820 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.289+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b81a1510 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.290+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f42a801ab60 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.290+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b819a6e0 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.290+0000 7f42aea00700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f429c0384d0 0x7f429c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.291+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f42a804c4c0 con 0x7f42b807adb0 2024-03-22T19:03:39.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.291+0000 7f42b5400700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f429c0384d0 0x7f429c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.291+0000 7f42b5400700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f429c0384d0 0x7f429c03a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42a4006fd0 tx=0x7f42a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.295+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f42a8018760 con 0x7f42b807adb0 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.521+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f42b802ccb0 con 0x7f42b807adb0 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.527+0000 7f42aea00700 1 -- 172.21.15.86:0/3202170317 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f42a800b3e0 con 0x7f42b807adb0 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.529+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f429c0384d0 msgr2=0x7f429c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.529+0000 7f42bd200700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f429c0384d0 0x7f429c03a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42a4006fd0 tx=0x7f42a4006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.529+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 msgr2=0x7f42b81a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.529+0000 7f42bd200700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b81a0930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42a8011670 tx=0x7f42a8011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.530+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 shutdown_connections 2024-03-22T19:03:39.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.530+0000 7f42bd200700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f429c0384d0 0x7f429c03a990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:39.927 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.530+0000 7f42bd200700 1 --2- 172.21.15.86:0/3202170317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42b807adb0 0x7f42b81a0930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:39.927 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.530+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 >> 172.21.15.86:0/3202170317 conn(0x7f42b8101ab0 msgr2=0x7f42b8197820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:39.927 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.530+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 shutdown_connections 2024-03-22T19:03:39.927 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:39.530+0000 7f42bd200700 1 -- 172.21.15.86:0/3202170317 wait complete. 2024-03-22T19:03:39.927 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-03-22T19:03:40.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:40 smithi086 ceph-mon[38032]: from='mgr.14120 172.21.15.86:0/262702567' entity='mgr.x' 2024-03-22T19:03:40.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:40 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3202170317' entity='client.admin' 2024-03-22T19:03:41.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:41 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1537228574' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-22T19:03:41.691 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:41 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: ignoring --setuser ceph since I am not root 2024-03-22T19:03:41.691 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:41 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: ignoring --setgroup ceph since I am not root 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.367+0000 7f22c8200700 1 Processor -- start 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.367+0000 7f22c8200700 1 -- start start 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.367+0000 7f22c8200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c00791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.367+0000 7f22c8200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c0079780 con 0x7f22c007ad50 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.368+0000 7f22c5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c00791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.368+0000 7f22c5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c00791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41906/0 (socket says 172.21.15.86:41906) 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.368+0000 7f22c5800700 1 -- 172.21.15.86:0/3576049835 learned_addr learned my addr 172.21.15.86:0/3576049835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:41.823 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.368+0000 7f22c5800700 1 -- 172.21.15.86:0/3576049835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00798c0 con 0x7f22c007ad50 2024-03-22T19:03:41.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.368+0000 7f22c5800700 1 --2- 172.21.15.86:0/3576049835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c00791b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22b40098a0 tx=0x7f22b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88f22c7982ff543 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:41.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.369+0000 7f22bfe00700 1 -- 172.21.15.86:0/3576049835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22b4004160 con 0x7f22c007ad50 2024-03-22T19:03:41.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.369+0000 7f22bfe00700 1 -- 172.21.15.86:0/3576049835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b40042c0 con 0x7f22c007ad50 2024-03-22T19:03:41.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.369+0000 7f22bfe00700 1 -- 172.21.15.86:0/3576049835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22b40119b0 con 0x7f22c007ad50 2024-03-22T19:03:41.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.369+0000 7f22c8200700 1 -- 172.21.15.86:0/3576049835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 msgr2=0x7f22c00791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.369+0000 7f22c8200700 1 --2- 172.21.15.86:0/3576049835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c00791b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22b40098a0 tx=0x7f22b4009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.370+0000 7f22c8200700 1 -- 172.21.15.86:0/3576049835 shutdown_connections 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.370+0000 7f22c8200700 1 --2- 172.21.15.86:0/3576049835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c00791b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.370+0000 7f22c8200700 1 -- 172.21.15.86:0/3576049835 >> 172.21.15.86:0/3576049835 conn(0x7f22c0101a50 msgr2=0x7f22c0103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.370+0000 7f22c8200700 1 -- 172.21.15.86:0/3576049835 shutdown_connections 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.370+0000 7f22c8200700 1 -- 172.21.15.86:0/3576049835 wait complete. 2024-03-22T19:03:41.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c8200700 1 Processor -- start 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c8200700 1 -- start start 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c8200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c8200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c019c9e0 con 0x7f22c007ad50 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41914/0 (socket says 172.21.15.86:41914) 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.371+0000 7f22c5800700 1 -- 172.21.15.86:0/1537228574 learned_addr learned my addr 172.21.15.86:0/1537228574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22c5800700 1 -- 172.21.15.86:0/1537228574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b4009550 con 0x7f22c007ad50 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22c5800700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c019c4a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22b4011670 tx=0x7f22b4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22b4003b20 con 0x7f22c007ad50 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b4003c80 con 0x7f22c007ad50 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c019cbe0 con 0x7f22c007ad50 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22b4019770 con 0x7f22c007ad50 2024-03-22T19:03:41.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.372+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c019d080 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.373+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45374+0+0 (secure 0 0 0) 0x7f22b401a430 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.373+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c010c930 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.374+0000 7f22be000700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f22a80384d0 0x7f22a803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.374+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f22b404c740 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.374+0000 7f22c4e00700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f22a80384d0 0x7f22a803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.375+0000 7f22c4e00700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f22a80384d0 0x7f22a803a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22ac006fd0 tx=0x7f22ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.379+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f22b402d9a0 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:40.649+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f22c010cb20 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.444+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 9) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f22b4022d90 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.445+0000 7f22be000700 1 -- 172.21.15.86:0/1537228574 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f22b40256f0 con 0x7f22c007ad50 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f22a80384d0 msgr2=0x7f22a803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f22a80384d0 0x7f22a803a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22ac006fd0 tx=0x7f22ac006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 msgr2=0x7f22c019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c019c4a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22b4011670 tx=0x7f22b4011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 shutdown_connections 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f22a80384d0 0x7f22a803a990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 --2- 172.21.15.86:0/1537228574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c007ad50 0x7f22c019c4a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.447+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 >> 172.21.15.86:0/1537228574 conn(0x7f22c0101a50 msgr2=0x7f22c0102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.448+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 shutdown_connections 2024-03-22T19:03:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:41.448+0000 7f22c8200700 1 -- 172.21.15.86:0/1537228574 wait complete. 2024-03-22T19:03:42.442 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:42 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:42.085+0000 7fde3a648200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T19:03:42.442 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:42 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:42.209+0000 7fde3a648200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T19:03:42.442 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:42 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:42.328+0000 7fde3a648200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T19:03:42.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:42 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1537228574' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-22T19:03:42.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:42 smithi086 ceph-mon[38032]: mgrmap e9: x(active, since 16s) 2024-03-22T19:03:42.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:42 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:42.441+0000 7fde3a648200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T19:03:42.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:42.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-22T19:03:42.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.278+0000 7f0c51c00700 1 Processor -- start 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.278+0000 7f0c51c00700 1 -- start start 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.278+0000 7f0c51c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.278+0000 7f0c51c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c079780 con 0x7f0c4c07ad50 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.279+0000 7f0c4be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.279+0000 7f0c4be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c0791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41944/0 (socket says 172.21.15.86:41944) 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.279+0000 7f0c4be00700 1 -- 172.21.15.86:0/429299122 learned_addr learned my addr 172.21.15.86:0/429299122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.279+0000 7f0c4be00700 1 -- 172.21.15.86:0/429299122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0798c0 con 0x7f0c4c07ad50 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.279+0000 7f0c4be00700 1 --2- 172.21.15.86:0/429299122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c0791b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0098a0 tx=0x7f0c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf47b79cfdb33bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:42.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c4aa00700 1 -- 172.21.15.86:0/429299122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c004160 con 0x7f0c4c07ad50 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c4aa00700 1 -- 172.21.15.86:0/429299122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c3c0042c0 con 0x7f0c4c07ad50 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c4aa00700 1 -- 172.21.15.86:0/429299122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c0119b0 con 0x7f0c4c07ad50 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c51c00700 1 -- 172.21.15.86:0/429299122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 msgr2=0x7f0c4c0791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c51c00700 1 --2- 172.21.15.86:0/429299122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c0791b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0098a0 tx=0x7f0c3c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c51c00700 1 -- 172.21.15.86:0/429299122 shutdown_connections 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c51c00700 1 --2- 172.21.15.86:0/429299122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c0791b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c51c00700 1 -- 172.21.15.86:0/429299122 >> 172.21.15.86:0/429299122 conn(0x7f0c4c1018c0 msgr2=0x7f0c4c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.280+0000 7f0c51c00700 1 -- 172.21.15.86:0/429299122 shutdown_connections 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.281+0000 7f0c51c00700 1 -- 172.21.15.86:0/429299122 wait complete. 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c51c00700 1 Processor -- start 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c51c00700 1 -- start start 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c51c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:42.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c51c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c19c7b0 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c4be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c4be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c19c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41956/0 (socket says 172.21.15.86:41956) 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.282+0000 7f0c4be00700 1 -- 172.21.15.86:0/1853757165 learned_addr learned my addr 172.21.15.86:0/1853757165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c4be00700 1 -- 172.21.15.86:0/1853757165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c009550 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c4be00700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c19c270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c011670 tx=0x7f0c3c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c003b20 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c19c9b0 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c3c003c80 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c019770 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.283+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c19ce50 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.284+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f0c3c019a30 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.284+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c10c760 con 0x7f0c4c07ad50 2024-03-22T19:03:42.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.285+0000 7f0c48c00700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 0x7f0c3003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.285+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0c3c04c790 con 0x7f0c4c07ad50 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.285+0000 7f0c4b400700 1 -- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 msgr2=0x7f0c3003a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.285+0000 7f0c4b400700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 0x7f0c3003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.290+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c3c018db0 con 0x7f0c4c07ad50 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.486+0000 7f0c4b400700 1 -- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 msgr2=0x7f0c3003a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.486+0000 7f0c4b400700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 0x7f0c3003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.556+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0c4c19fb00 con 0x7f0c4c07ad50 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.556+0000 7f0c48c00700 1 -- 172.21.15.86:0/1853757165 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0c3c018350 con 0x7f0c4c07ad50 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 msgr2=0x7f0c3003a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 0x7f0c3003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 msgr2=0x7f0c4c19c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c19c270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c011670 tx=0x7f0c3c011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 shutdown_connections 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f0c30038520 0x7f0c3003a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 --2- 172.21.15.86:0/1853757165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c4c07ad50 0x7f0c4c19c270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 >> 172.21.15.86:0/1853757165 conn(0x7f0c4c1018c0 msgr2=0x7f0c4c102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 shutdown_connections 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:42.559+0000 7f0c51c00700 1 -- 172.21.15.86:0/1853757165 wait complete. 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-03-22T19:03:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-03-22T19:03:43.444 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:43 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:43.315+0000 7fde3a648200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:03:43.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:43 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1853757165' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T19:03:43.745 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:43 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:43.635+0000 7fde3a648200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T19:03:43.745 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:43 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:43.744+0000 7fde3a648200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T19:03:44.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:43 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:43.881+0000 7fde3a648200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T19:03:44.354 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:44 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:44.076+0000 7fde3a648200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T19:03:44.354 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:44 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:44.242+0000 7fde3a648200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T19:03:44.758 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:44 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:44.353+0000 7fde3a648200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T19:03:45.258 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:44 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:44.814+0000 7fde3a648200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T19:03:45.661 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:45 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:45.294+0000 7fde3a648200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T19:03:45.662 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:45 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:45.540+0000 7fde3a648200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-22T19:03:46.912 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:45 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:45.660+0000 7fde3a648200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T19:03:46.913 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:46 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:46.594+0000 7fde3a648200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T19:03:46.913 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:46 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:46.717+0000 7fde3a648200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T19:03:46.913 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:46 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:46.911+0000 7fde3a648200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T19:03:48.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:47 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:47.712+0000 7fde3a648200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-22T19:03:48.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:47 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:47.952+0000 7fde3a648200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T19:03:48.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:48 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:48.205+0000 7fde3a648200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T19:03:49.707 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:49 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:49.332+0000 7fde3a648200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T19:03:50.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:49 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:49.706+0000 7fde3a648200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:03:50.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:49 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:49.827+0000 7fde3a648200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T19:03:50.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:03:50.080+0000 7fde3a648200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T19:03:50.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: Active manager daemon x restarted 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: Activating manager daemon x 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: osdmap e3: 0 total, 0 up, 0 in 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: mgrmap e10: x(active, starting, since 0.00392323s) 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: Manager daemon x is now available 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:50.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:03:51.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-22T19:03:51.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-22T19:03:51.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:51 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:50] ENGINE Bus STARTING 2024-03-22T19:03:51.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:51 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:50] ENGINE Serving on http://172.21.15.86:8765 2024-03-22T19:03:51.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:51 smithi086 ceph-mon[38032]: mgrmap e11: x(active, since 1.00789s) 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752fa00700 1 Processor -- start 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752fa00700 1 -- start start 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f7528106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7528106ee0 con 0x7f75281064f0 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f7528106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f7528106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41958/0 (socket says 172.21.15.86:41958) 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.369+0000 7f752d000700 1 -- 172.21.15.86:0/1307105079 learned_addr learned my addr 172.21.15.86:0/1307105079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:51.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.370+0000 7f752d000700 1 -- 172.21.15.86:0/1307105079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75281076f0 con 0x7f75281064f0 2024-03-22T19:03:51.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.370+0000 7f752d000700 1 --2- 172.21.15.86:0/1307105079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f7528106910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098a0 tx=0x7f751c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc8cafa4fea3459 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:51.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.370+0000 7f7527400700 1 -- 172.21.15.86:0/1307105079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f751c004160 con 0x7f75281064f0 2024-03-22T19:03:51.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.370+0000 7f7527400700 1 -- 172.21.15.86:0/1307105079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f751c0042c0 con 0x7f75281064f0 2024-03-22T19:03:51.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.370+0000 7f7527400700 1 -- 172.21.15.86:0/1307105079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f751c0119b0 con 0x7f75281064f0 2024-03-22T19:03:51.627 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 -- 172.21.15.86:0/1307105079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 msgr2=0x7f7528106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 --2- 172.21.15.86:0/1307105079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f7528106910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098a0 tx=0x7f751c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 -- 172.21.15.86:0/1307105079 shutdown_connections 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 --2- 172.21.15.86:0/1307105079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f7528106910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 -- 172.21.15.86:0/1307105079 >> 172.21.15.86:0/1307105079 conn(0x7f7528101a50 msgr2=0x7f7528103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 -- 172.21.15.86:0/1307105079 shutdown_connections 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.371+0000 7f752fa00700 1 -- 172.21.15.86:0/1307105079 wait complete. 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.373+0000 7f752fa00700 1 Processor -- start 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.373+0000 7f752fa00700 1 -- start start 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.374+0000 7f752fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f752819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.374+0000 7f752fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752819a7f0 con 0x7f75281064f0 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.374+0000 7f752d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f752819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.374+0000 7f752d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f752819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41974/0 (socket says 172.21.15.86:41974) 2024-03-22T19:03:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.374+0000 7f752d000700 1 -- 172.21.15.86:0/2753952405 learned_addr learned my addr 172.21.15.86:0/2753952405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.374+0000 7f752d000700 1 -- 172.21.15.86:0/2753952405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009550 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.375+0000 7f752d000700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f752819a2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f751c003e60 tx=0x7f751c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.375+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f751c003a60 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.375+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f751c003bc0 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.375+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f751c0196c0 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.375+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752819a9f0 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.375+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752819ae90 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.376+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f751c01ab60 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.376+0000 7f7525600700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.377+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 --> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f751003b120 con 0x7f7510038550 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.377+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f751c04c660 con 0x7f75281064f0 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.377+0000 7f7527e00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.377+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:03:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.578+0000 7f7527e00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.578+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.979+0000 7f7527e00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:43.979+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:44.780+0000 7f7527e00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:44.780+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:46.381+0000 7f7527e00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:46.382+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:49.585+0000 7f7527e00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2748771516 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:49.585+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:50.085+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f751c01a7b0 con 0x7f75281064f0 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:50.086+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 msgr2=0x7f751003aa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:03:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:50.086+0000 7f7525600700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.088+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f751c02acf0 con 0x7f75281064f0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.088+0000 7f7525600700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f751003bfc0 0x7f751003e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.088+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f751003b120 con 0x7f751003bfc0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.089+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f751003bfc0 0x7f751003e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.090+0000 7f7527e00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f751003bfc0 0x7f751003e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7514003a10 tx=0x7f7514009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.091+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f751003b120 con 0x7f751003bfc0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.097+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f752810a920 con 0x7f751003bfc0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f7525600700 1 -- 172.21.15.86:0/2753952405 <== mgr.14152 v2:172.21.15.86:6800/1126645338 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f752810a920 con 0x7f751003bfc0 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f751003bfc0 msgr2=0x7f751003e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f751003bfc0 0x7f751003e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7514003a10 tx=0x7f7514009270 comp rx=0 tx=0).stop 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 msgr2=0x7f752819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f752819a2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f751c003e60 tx=0x7f751c011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 shutdown_connections 2024-03-22T19:03:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75281064f0 0x7f752819a2b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f751003bfc0 0x7f751003e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 --2- 172.21.15.86:0/2753952405 >> [v2:172.21.15.86:6800/2748771516,v1:172.21.15.86:6801/2748771516] conn(0x7f7510038550 0x7f751003aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.098+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 >> 172.21.15.86:0/2753952405 conn(0x7f7528101a50 msgr2=0x7f75281026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.099+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 shutdown_connections 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:51.099+0000 7f752fa00700 1 -- 172.21.15.86:0/2753952405 wait complete. 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-03-22T19:03:51.632 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-03-22T19:03:52.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:52 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:50] ENGINE Serving on https://172.21.15.86:7150 2024-03-22T19:03:52.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:52 smithi086 ceph-mon[38032]: [22/Mar/2024:19:03:50] ENGINE Bus STARTED 2024-03-22T19:03:52.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:52 smithi086 ceph-mon[38032]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T19:03:52.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:52 smithi086 ceph-mon[38032]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T19:03:52.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.037+0000 7f9a49c00700 1 Processor -- start 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.037+0000 7f9a49c00700 1 -- start start 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.038+0000 7f9a49c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a44109440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.038+0000 7f9a49c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44074410 con 0x7f9a44107050 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.038+0000 7f9a42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a44109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:52.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.038+0000 7f9a42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a44109440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58286/0 (socket says 172.21.15.86:58286) 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.038+0000 7f9a42a00700 1 -- 172.21.15.86:0/1512739708 learned_addr learned my addr 172.21.15.86:0/1512739708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.038+0000 7f9a42a00700 1 -- 172.21.15.86:0/1512739708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44074550 con 0x7f9a44107050 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.039+0000 7f9a42a00700 1 --2- 172.21.15.86:0/1512739708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a44109440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a340098a0 tx=0x7f9a34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab0fccf488ab34b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.039+0000 7f9a41600700 1 -- 172.21.15.86:0/1512739708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a34004160 con 0x7f9a44107050 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.039+0000 7f9a41600700 1 -- 172.21.15.86:0/1512739708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a340042c0 con 0x7f9a44107050 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.039+0000 7f9a41600700 1 -- 172.21.15.86:0/1512739708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a340119b0 con 0x7f9a44107050 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 -- 172.21.15.86:0/1512739708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 msgr2=0x7f9a44109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 --2- 172.21.15.86:0/1512739708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a44109440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a340098a0 tx=0x7f9a34009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 -- 172.21.15.86:0/1512739708 shutdown_connections 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 --2- 172.21.15.86:0/1512739708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a44109440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 -- 172.21.15.86:0/1512739708 >> 172.21.15.86:0/1512739708 conn(0x7f9a441009c0 msgr2=0x7f9a44102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 -- 172.21.15.86:0/1512739708 shutdown_connections 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.040+0000 7f9a49c00700 1 -- 172.21.15.86:0/1512739708 wait complete. 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.041+0000 7f9a49c00700 1 Processor -- start 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a49c00700 1 -- start start 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a49c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a4419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a49c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4419a830 con 0x7f9a44107050 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a4419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a4419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58302/0 (socket says 172.21.15.86:58302) 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a42a00700 1 -- 172.21.15.86:0/3061196896 learned_addr learned my addr 172.21.15.86:0/3061196896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.042+0000 7f9a42a00700 1 -- 172.21.15.86:0/3061196896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34009550 con 0x7f9a44107050 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.043+0000 7f9a42a00700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a4419a2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a34011980 tx=0x7f9a34011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.043+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a34003af0 con 0x7f9a44107050 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.043+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a34003c50 con 0x7f9a44107050 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.043+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4419aa30 con 0x7f9a44107050 2024-03-22T19:03:52.855 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.043+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a34019710 con 0x7f9a44107050 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.043+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4419aed0 con 0x7f9a44107050 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.044+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f9a3401a430 con 0x7f9a44107050 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.045+0000 7f9a3b400700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9a280384a0 0x7f9a2803a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.045+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a3404c5a0 con 0x7f9a44107050 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.045+0000 7f9a42000700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9a280384a0 0x7f9a2803a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.045+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a440623e0 con 0x7f9a44107050 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.046+0000 7f9a42000700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9a280384a0 0x7f9a2803a960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006fd0 tx=0x7f9a30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.051+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9a3402d9a0 con 0x7f9a44107050 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.286+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a44101c80 con 0x7f9a280384a0 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.401+0000 7f9a3b400700 1 -- 172.21.15.86:0/3061196896 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9a44101c80 con 0x7f9a280384a0 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9a280384a0 msgr2=0x7f9a2803a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9a280384a0 0x7f9a2803a960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006fd0 tx=0x7f9a30006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 msgr2=0x7f9a4419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a4419a2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a34011980 tx=0x7f9a34011a60 comp rx=0 tx=0).stop 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 shutdown_connections 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9a280384a0 0x7f9a2803a960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 --2- 172.21.15.86:0/3061196896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a44107050 0x7f9a4419a2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 >> 172.21.15.86:0/3061196896 conn(0x7f9a441009c0 msgr2=0x7f9a44101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 shutdown_connections 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:52.404+0000 7f9a49c00700 1 -- 172.21.15.86:0/3061196896 wait complete. 2024-03-22T19:03:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-03-22T19:03:53.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:53 smithi086 ceph-mon[38032]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:53.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:03:53.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:03:53.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:53 smithi086 ceph-mon[38032]: mgrmap e12: x(active, since 2s) 2024-03-22T19:03:53.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:03:54.428 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fSrTr3C76s0fz6IO9dSB0OM7YWp74Soa1OddMcXupoqszeQmIkh9q", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711134233, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-22T19:03:54.428 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.315+0000 7f9dc8800700 1 Processor -- start 2024-03-22T19:03:54.428 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.315+0000 7f9dc8800700 1 -- start start 2024-03-22T19:03:54.428 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.315+0000 7f9dc8800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc0079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.315+0000 7f9dc8800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0079760 con 0x7f9dc007ad30 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.316+0000 7f9dc5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.316+0000 7f9dc5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc0079190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58316/0 (socket says 172.21.15.86:58316) 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.316+0000 7f9dc5e00700 1 -- 172.21.15.86:0/926345465 learned_addr learned my addr 172.21.15.86:0/926345465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.316+0000 7f9dc5e00700 1 -- 172.21.15.86:0/926345465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00798a0 con 0x7f9dc007ad30 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.317+0000 7f9dc5e00700 1 --2- 172.21.15.86:0/926345465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc0079190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9db40098a0 tx=0x7f9db4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75aea308623186 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.317+0000 7f9dc4a00700 1 -- 172.21.15.86:0/926345465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9db4004160 con 0x7f9dc007ad30 2024-03-22T19:03:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.317+0000 7f9dc4a00700 1 -- 172.21.15.86:0/926345465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db40042c0 con 0x7f9dc007ad30 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.317+0000 7f9dc4a00700 1 -- 172.21.15.86:0/926345465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9db40119b0 con 0x7f9dc007ad30 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.318+0000 7f9dc8800700 1 -- 172.21.15.86:0/926345465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 msgr2=0x7f9dc0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.318+0000 7f9dc8800700 1 --2- 172.21.15.86:0/926345465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc0079190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9db40098a0 tx=0x7f9db4009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.318+0000 7f9dc8800700 1 -- 172.21.15.86:0/926345465 shutdown_connections 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.318+0000 7f9dc8800700 1 --2- 172.21.15.86:0/926345465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc0079190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.318+0000 7f9dc8800700 1 -- 172.21.15.86:0/926345465 >> 172.21.15.86:0/926345465 conn(0x7f9dc0101ab0 msgr2=0x7f9dc0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.318+0000 7f9dc8800700 1 -- 172.21.15.86:0/926345465 shutdown_connections 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.319+0000 7f9dc8800700 1 -- 172.21.15.86:0/926345465 wait complete. 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.319+0000 7f9dc8800700 1 Processor -- start 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc8800700 1 -- start start 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc8800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc8800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc019c9e0 con 0x7f9dc007ad30 2024-03-22T19:03:54.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58318/0 (socket says 172.21.15.86:58318) 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc5e00700 1 -- 172.21.15.86:0/2902038981 learned_addr learned my addr 172.21.15.86:0/2902038981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.320+0000 7f9dc5e00700 1 -- 172.21.15.86:0/2902038981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009550 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.321+0000 7f9dc5e00700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc019c4a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9db4011980 tx=0x7f9db4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.321+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9db4003af0 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.321+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db4003c50 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.321+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9db4019760 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.321+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc019cbe0 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.321+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc019d080 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.322+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f9db4019950 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.323+0000 7f9dbaa00700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9da8038570 0x7f9da803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.323+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9db404c9a0 con 0x7f9dc007ad30 2024-03-22T19:03:54.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.323+0000 7f9dc5400700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9da8038570 0x7f9da803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.323+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc00623e0 con 0x7f9dc007ad30 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.324+0000 7f9dc5400700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9da8038570 0x7f9da803aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9db0006fd0 tx=0x7f9db0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.328+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9db4018350 con 0x7f9dc007ad30 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.584+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9dc019f9d0 con 0x7f9da8038570 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.813+0000 7f9dbaa00700 1 -- 172.21.15.86:0/2902038981 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f9dc019f9d0 con 0x7f9da8038570 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9da8038570 msgr2=0x7f9da803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9da8038570 0x7f9da803aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9db0006fd0 tx=0x7f9db0006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 msgr2=0x7f9dc019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc019c4a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9db4011980 tx=0x7f9db4011a60 comp rx=0 tx=0).stop 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 shutdown_connections 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9da8038570 0x7f9da803aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:54.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 --2- 172.21.15.86:0/2902038981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc007ad30 0x7f9dc019c4a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:54.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 >> 172.21.15.86:0/2902038981 conn(0x7f9dc0101ab0 msgr2=0x7f9dc0102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:54.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 shutdown_connections 2024-03-22T19:03:54.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:53.816+0000 7f9dc8800700 1 -- 172.21.15.86:0/2902038981 wait complete. 2024-03-22T19:03:54.433 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-03-22T19:03:54.433 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:54 smithi086 ceph-mon[38032]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:03:54.433 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:03:55.379 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-03-22T19:03:55.379 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.813+0000 7f70b1800700 1 Processor -- start 2024-03-22T19:03:55.379 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.814+0000 7f70b1800700 1 -- start start 2024-03-22T19:03:55.379 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.814+0000 7f70b1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:55.379 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.814+0000 7f70b1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac079780 con 0x7f70ac07ad50 2024-03-22T19:03:55.379 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.814+0000 7f70aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.814+0000 7f70aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac0791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58326/0 (socket says 172.21.15.86:58326) 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.814+0000 7f70aaa00700 1 -- 172.21.15.86:0/3754066229 learned_addr learned my addr 172.21.15.86:0/3754066229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.815+0000 7f70aaa00700 1 -- 172.21.15.86:0/3754066229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac0798c0 con 0x7f70ac07ad50 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.815+0000 7f70aaa00700 1 --2- 172.21.15.86:0/3754066229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac0791b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098a0 tx=0x7f709c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b54a1d9956e501 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.815+0000 7f70a9600700 1 -- 172.21.15.86:0/3754066229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f709c004160 con 0x7f70ac07ad50 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.815+0000 7f70a9600700 1 -- 172.21.15.86:0/3754066229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c0042c0 con 0x7f70ac07ad50 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.815+0000 7f70a9600700 1 -- 172.21.15.86:0/3754066229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f709c0119b0 con 0x7f70ac07ad50 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 -- 172.21.15.86:0/3754066229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 msgr2=0x7f70ac0791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 --2- 172.21.15.86:0/3754066229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac0791b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098a0 tx=0x7f709c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 -- 172.21.15.86:0/3754066229 shutdown_connections 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 --2- 172.21.15.86:0/3754066229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac0791b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 -- 172.21.15.86:0/3754066229 >> 172.21.15.86:0/3754066229 conn(0x7f70ac101a50 msgr2=0x7f70ac103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 -- 172.21.15.86:0/3754066229 shutdown_connections 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.816+0000 7f70b1800700 1 -- 172.21.15.86:0/3754066229 wait complete. 2024-03-22T19:03:55.380 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70b1800700 1 Processor -- start 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70b1800700 1 -- start start 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70b1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70b1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac19c9e0 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58336/0 (socket says 172.21.15.86:58336) 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.817+0000 7f70aaa00700 1 -- 172.21.15.86:0/940113643 learned_addr learned my addr 172.21.15.86:0/940113643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70aaa00700 1 -- 172.21.15.86:0/940113643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009550 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70aaa00700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac19c4a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f709c011670 tx=0x7f709c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f709c003b20 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c003c80 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f709c019770 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac19cbe0 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.818+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac19d080 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.820+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac10c930 con 0x7f70ac07ad50 2024-03-22T19:03:55.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.820+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f709c01ab60 con 0x7f70ac07ad50 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.820+0000 7f70a3400700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7090038570 0x7f709003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.820+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f709c04c860 con 0x7f70ac07ad50 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.820+0000 7f70aa000700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7090038570 0x7f709003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.821+0000 7f70aa000700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7090038570 0x7f709003aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7098006fd0 tx=0x7f7098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:54.825+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f709c01a430 con 0x7f70ac07ad50 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.022+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f70ac19fce0 con 0x7f70ac07ad50 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.023+0000 7f70a3400700 1 -- 172.21.15.86:0/940113643 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f709c02db00 con 0x7f70ac07ad50 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7090038570 msgr2=0x7f709003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7090038570 0x7f709003aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7098006fd0 tx=0x7f7098006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 msgr2=0x7f70ac19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac19c4a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f709c011670 tx=0x7f709c011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 shutdown_connections 2024-03-22T19:03:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7090038570 0x7f709003aa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:55.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 --2- 172.21.15.86:0/940113643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70ac07ad50 0x7f70ac19c4a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:55.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.025+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 >> 172.21.15.86:0/940113643 conn(0x7f70ac101a50 msgr2=0x7f70ac102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:55.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.026+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 shutdown_connections 2024-03-22T19:03:55.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.026+0000 7f70b1800700 1 -- 172.21.15.86:0/940113643 wait complete. 2024-03-22T19:03:55.395 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T19:03:55.395 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-22T19:03:55.410 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T19:03:55.410 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-22T19:03:55.410 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-22T19:03:55.410 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-22T19:03:55.413 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-03-22T19:03:55.413 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:55.413 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086.front.sepia.ceph.com:8443/ 2024-03-22T19:03:55.413 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-03-22T19:03:55.413 INFO:teuthology.orchestra.run.smithi086.stdout: Password: h0dkd53mf6 2024-03-22T19:03:55.413 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:55.414 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config directory 2024-03-22T19:03:55.414 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling autotune for osd_memory_target 2024-03-22T19:03:55.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:55 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/940113643' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-22T19:03:56.455 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.831+0000 7f4ba1200700 1 Processor -- start 2024-03-22T19:03:56.455 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.831+0000 7f4ba1200700 1 -- start start 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.832+0000 7f4ba1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.832+0000 7f4ba1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c106f00 con 0x7f4b9c106510 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.832+0000 7f4b99e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.832+0000 7f4b99e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58344/0 (socket says 172.21.15.86:58344) 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.832+0000 7f4b99e00700 1 -- 172.21.15.86:0/571764322 learned_addr learned my addr 172.21.15.86:0/571764322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.833+0000 7f4b99e00700 1 -- 172.21.15.86:0/571764322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c107710 con 0x7f4b9c106510 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.833+0000 7f4b99e00700 1 --2- 172.21.15.86:0/571764322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c106930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098a0 tx=0x7f4b8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27f0daf1cbd45de server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.833+0000 7f4b98a00700 1 -- 172.21.15.86:0/571764322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c004160 con 0x7f4b9c106510 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.833+0000 7f4b98a00700 1 -- 172.21.15.86:0/571764322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c0042c0 con 0x7f4b9c106510 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.833+0000 7f4b98a00700 1 -- 172.21.15.86:0/571764322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c0119b0 con 0x7f4b9c106510 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 -- 172.21.15.86:0/571764322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 msgr2=0x7f4b9c106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 --2- 172.21.15.86:0/571764322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c106930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098a0 tx=0x7f4b8c009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 -- 172.21.15.86:0/571764322 shutdown_connections 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 --2- 172.21.15.86:0/571764322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c106930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 -- 172.21.15.86:0/571764322 >> 172.21.15.86:0/571764322 conn(0x7f4b9c101ab0 msgr2=0x7f4b9c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 -- 172.21.15.86:0/571764322 shutdown_connections 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.834+0000 7f4ba1200700 1 -- 172.21.15.86:0/571764322 wait complete. 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4ba1200700 1 Processor -- start 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4ba1200700 1 -- start start 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4ba1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4ba1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c19ca90 con 0x7f4b9c106510 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4b99e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4b99e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58360/0 (socket says 172.21.15.86:58360) 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.835+0000 7f4b99e00700 1 -- 172.21.15.86:0/2552029858 learned_addr learned my addr 172.21.15.86:0/2552029858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4b99e00700 1 -- 172.21.15.86:0/2552029858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c009550 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4b99e00700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c19c550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011670 tx=0x7f4b8c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c003af0 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c003c50 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c024820 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c19cc90 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.836+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c19d130 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.837+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f4b8c01ab60 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.838+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c10a940 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.838+0000 7f4b92a00700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4b80038570 0x7f4b8003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.838+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4b8c04c5d0 con 0x7f4b9c106510 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.838+0000 7f4b99400700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4b80038570 0x7f4b8003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:56.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.839+0000 7f4b99400700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4b80038570 0x7f4b8003aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b88006fd0 tx=0x7f4b88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:55.843+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4b8c018760 con 0x7f4b9c106510 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.043+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4b9c10ab30 con 0x7f4b9c106510 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.043+0000 7f4b92a00700 1 -- 172.21.15.86:0/2552029858 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4b8c00b3e0 con 0x7f4b9c106510 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4b80038570 msgr2=0x7f4b8003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4b80038570 0x7f4b8003aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b88006fd0 tx=0x7f4b88006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 msgr2=0x7f4b9c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c19c550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011670 tx=0x7f4b8c011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 shutdown_connections 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4b80038570 0x7f4b8003aa30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 --2- 172.21.15.86:0/2552029858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9c106510 0x7f4b9c19c550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 >> 172.21.15.86:0/2552029858 conn(0x7f4b9c101ab0 msgr2=0x7f4b9c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 shutdown_connections 2024-03-22T19:03:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.046+0000 7f4ba1200700 1 -- 172.21.15.86:0/2552029858 wait complete. 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.831+0000 7f2fa5000700 1 Processor -- start 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.831+0000 7f2fa5000700 1 -- start start 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.831+0000 7f2fa5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa0106740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.831+0000 7f2fa5000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0106d10 con 0x7f2fa0106320 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.832+0000 7f2f9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa0106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.832+0000 7f2f9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa0106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58374/0 (socket says 172.21.15.86:58374) 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.832+0000 7f2f9f400700 1 -- 172.21.15.86:0/3134324126 learned_addr learned my addr 172.21.15.86:0/3134324126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.832+0000 7f2f9f400700 1 -- 172.21.15.86:0/3134324126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0107520 con 0x7f2fa0106320 2024-03-22T19:03:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.832+0000 7f2f9f400700 1 --2- 172.21.15.86:0/3134324126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa0106740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f900098a0 tx=0x7f2f90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659d7aab754ce88d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.832+0000 7f2f9e000700 1 -- 172.21.15.86:0/3134324126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f90004160 con 0x7f2fa0106320 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2f9e000700 1 -- 172.21.15.86:0/3134324126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f900042c0 con 0x7f2fa0106320 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2f9e000700 1 -- 172.21.15.86:0/3134324126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f900119b0 con 0x7f2fa0106320 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 -- 172.21.15.86:0/3134324126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 msgr2=0x7f2fa0106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 --2- 172.21.15.86:0/3134324126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa0106740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f900098a0 tx=0x7f2f90009bb0 comp rx=0 tx=0).stop 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 -- 172.21.15.86:0/3134324126 shutdown_connections 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 --2- 172.21.15.86:0/3134324126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa0106740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 -- 172.21.15.86:0/3134324126 >> 172.21.15.86:0/3134324126 conn(0x7f2fa01018c0 msgr2=0x7f2fa0103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 -- 172.21.15.86:0/3134324126 shutdown_connections 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.833+0000 7f2fa5000700 1 -- 172.21.15.86:0/3134324126 wait complete. 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.834+0000 7f2fa5000700 1 Processor -- start 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.834+0000 7f2fa5000700 1 -- start start 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2fa5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2fa5000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019a5f0 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2f9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2f9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58386/0 (socket says 172.21.15.86:58386) 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2f9f400700 1 -- 172.21.15.86:0/3915034406 learned_addr learned my addr 172.21.15.86:0/3915034406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2f9f400700 1 -- 172.21.15.86:0/3915034406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90009550 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.835+0000 7f2f9f400700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa019a0b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f90003e60 tx=0x7f2f90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.836+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f90003a60 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.836+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa019a7f0 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.836+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f90003bc0 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.836+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f900196c0 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.836+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa019ac90 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.837+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa010a750 con 0x7f2fa0106320 2024-03-22T19:03:57.502 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.837+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f2f9001ab60 con 0x7f2fa0106320 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.837+0000 7f2f97e00700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2f840385a0 0x7f2f8403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.837+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f9004c8e0 con 0x7f2fa0106320 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.837+0000 7f2f9ea00700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2f840385a0 0x7f2f8403aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.838+0000 7f2f9ea00700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2f840385a0 0x7f2f8403aa60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:56.842+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2f90019980 con 0x7f2fa0106320 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.149+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2fa010a940 con 0x7f2fa0106320 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.152+0000 7f2f97e00700 1 -- 172.21.15.86:0/3915034406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f2f90018350 con 0x7f2fa0106320 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2f840385a0 msgr2=0x7f2f8403aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2f840385a0 0x7f2f8403aa60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).stop 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 msgr2=0x7f2fa019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa019a0b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f90003e60 tx=0x7f2f90011a30 comp rx=0 tx=0).stop 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 shutdown_connections 2024-03-22T19:03:57.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2f840385a0 0x7f2f8403aa60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 --2- 172.21.15.86:0/3915034406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa0106320 0x7f2fa019a0b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 >> 172.21.15.86:0/3915034406 conn(0x7f2fa01018c0 msgr2=0x7f2fa01024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.154+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 shutdown_connections 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-22T19:03:57.155+0000 7f2fa5000700 1 -- 172.21.15.86:0/3915034406 wait complete. 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.504 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:03:57.505 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-03-22T19:03:57.557 INFO:tasks.cephadm:Fetching config... 2024-03-22T19:03:57.558 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:03:57.558 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-22T19:03:57.598 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-22T19:03:57.599 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:03:57.599 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-22T19:03:57.680 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-22T19:03:57.680 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:03:57.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-22T19:03:57.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:57 smithi086 ceph-mon[38032]: mgrmap e13: x(active, since 6s) 2024-03-22T19:03:57.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:03:57 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3915034406' entity='client.admin' 2024-03-22T19:03:57.793 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-22T19:03:57.793 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:03:57.793 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-22T19:03:57.877 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-22T19:03:57.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T19:03:58.008 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:58.038 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T19:03:58.084 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:58.097 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T19:03:58.146 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZOq6jqQtZ9RJIokeUEYNpRyQNIsq8c/V2XKp4sI/COrlhZLl2s+NCFb5hfGc1gR0Jst+FvuAEbe2aAXBhOLR7Shm/dxjVry3lt69nfs/6nUGlaJYUvnTpR1fSSsYtRknid1fEGzf0KodR/C/DTNiP+9ux/y+oxXDcUcIDzx1hG7/JCwnNxPfnFCjIC6GXQ0SaD2CxLiB9YN5u/KsAI94I9mHcLi70/i6GOsCW4DVbMDWFwmdXW/d4VPommD7GHHM3dLm944arkiRNTN6TKGPwgvcWOc+NwbEYcyRqEcvz96ASpxMaqaENJFDvzV39ZLn0qSvYDId+BjUvFJoZTCnNglaYpEyttNmc5nqu+um9UVnnJ9WtlbFR0+Ti8XA4Dd71T4cSmgncZ34jqldhBT0s9HpoQjv0NYenaZ5+vt2hB6I9LMB36I1LN2d/cpMmBsgq7F0X6gBlfh+P6Fd4OfiCBBpBRKSTZS30IoGIWnX51EzAMxO/xq1kqZ5uX5RCIbM= ceph-b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:03:58.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-22T19:03:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:03:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.867+0000 7f2106800700 1 -- 172.21.15.86:0/589098506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 msgr2=0x7f21000731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.867+0000 7f2106800700 1 --2- 172.21.15.86:0/589098506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f21000731e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009910 tx=0x7f20f0009c20 comp rx=0 tx=0).stop 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.867+0000 7f2106800700 1 -- 172.21.15.86:0/589098506 shutdown_connections 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.867+0000 7f2106800700 1 --2- 172.21.15.86:0/589098506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f21000731e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.867+0000 7f2106800700 1 -- 172.21.15.86:0/589098506 >> 172.21.15.86:0/589098506 conn(0x7f21000fbcb0 msgr2=0x7f21000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.868+0000 7f2106800700 1 -- 172.21.15.86:0/589098506 shutdown_connections 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.868+0000 7f2106800700 1 -- 172.21.15.86:0/589098506 wait complete. 2024-03-22T19:03:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.868+0000 7f2106800700 1 Processor -- start 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f2106800700 1 -- start start 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f2106800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f2100197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f2106800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100198290 con 0x7f2100074d80 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f20ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f2100197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f20ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f2100197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55012/0 (socket says 172.21.15.86:55012) 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f20ff400700 1 -- 172.21.15.86:0/1974680086 learned_addr learned my addr 172.21.15.86:0/1974680086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.869+0000 7f20ff400700 1 -- 172.21.15.86:0/1974680086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00095f0 con 0x7f2100074d80 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.870+0000 7f20ff400700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f2100197d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20f0005950 tx=0x7f20f00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.870+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20f001d070 con 0x7f2100074d80 2024-03-22T19:03:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.870+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2100198490 con 0x7f2100074d80 2024-03-22T19:03:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.870+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f000bc10 con 0x7f2100074d80 2024-03-22T19:03:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.870+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20f00039d0 con 0x7f2100074d80 2024-03-22T19:03:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.870+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2100198930 con 0x7f2100074d80 2024-03-22T19:03:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.871+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f20f0003c90 con 0x7f2100074d80 2024-03-22T19:03:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.871+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2100191630 con 0x7f2100074d80 2024-03-22T19:03:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.872+0000 7f20f7e00700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20e4038570 0x7f20e403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:03:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.872+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f20f004c300 con 0x7f2100074d80 2024-03-22T19:03:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.872+0000 7f20fea00700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20e4038570 0x7f20e403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:03:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.872+0000 7f20fea00700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20e4038570 0x7f20e403aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006fd0 tx=0x7f20ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:03:59.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:03:59.876+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f20f000ba70 con 0x7f2100074d80 2024-03-22T19:04:00.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.105+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f21000623e0 con 0x7f2100074d80 2024-03-22T19:04:00.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.109+0000 7f20f7e00700 1 -- 172.21.15.86:0/1974680086 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f20f0029360 con 0x7f2100074d80 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20e4038570 msgr2=0x7f20e403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20e4038570 0x7f20e403aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006fd0 tx=0x7f20ec006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 msgr2=0x7f2100197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f2100197d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20f0005950 tx=0x7f20f00057c0 comp rx=0 tx=0).stop 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 shutdown_connections 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20e4038570 0x7f20e403aa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 --2- 172.21.15.86:0/1974680086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2100074d80 0x7f2100197d50 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20f0005950 tx=0x7f20f00057c0 comp rx=0 tx=0).stop 2024-03-22T19:04:00.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 >> 172.21.15.86:0/1974680086 conn(0x7f21000fbcb0 msgr2=0x7f21000fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:00.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 shutdown_connections 2024-03-22T19:04:00.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:00.112+0000 7f2106800700 1 -- 172.21.15.86:0/1974680086 wait complete. 2024-03-22T19:04:00.604 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-22T19:04:00.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-22T19:04:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1974680086' entity='client.admin' 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:01.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.167+0000 7fd157c00700 1 -- 172.21.15.86:0/2106427079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 msgr2=0x7fd1500731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.167+0000 7fd157c00700 1 --2- 172.21.15.86:0/2106427079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd1500731c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd144009910 tx=0x7fd144009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.167+0000 7fd157c00700 1 -- 172.21.15.86:0/2106427079 shutdown_connections 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.167+0000 7fd157c00700 1 --2- 172.21.15.86:0/2106427079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd1500731c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.167+0000 7fd157c00700 1 -- 172.21.15.86:0/2106427079 >> 172.21.15.86:0/2106427079 conn(0x7fd1500fbcb0 msgr2=0x7fd1500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.167+0000 7fd157c00700 1 -- 172.21.15.86:0/2106427079 shutdown_connections 2024-03-22T19:04:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.168+0000 7fd157c00700 1 -- 172.21.15.86:0/2106427079 wait complete. 2024-03-22T19:04:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.169+0000 7fd157c00700 1 Processor -- start 2024-03-22T19:04:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.169+0000 7fd157c00700 1 -- start start 2024-03-22T19:04:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.169+0000 7fd157c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd150197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.170+0000 7fd157c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd150198290 con 0x7fd150074d60 2024-03-22T19:04:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.170+0000 7fd155200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd150197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.170+0000 7fd155200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd150197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55032/0 (socket says 172.21.15.86:55032) 2024-03-22T19:04:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.170+0000 7fd155200700 1 -- 172.21.15.86:0/3204929737 learned_addr learned my addr 172.21.15.86:0/3204929737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.170+0000 7fd155200700 1 -- 172.21.15.86:0/3204929737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440095f0 con 0x7fd150074d60 2024-03-22T19:04:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.170+0000 7fd155200700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd150197d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd144005950 tx=0x7fd1440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.171+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd14401d070 con 0x7fd150074d60 2024-03-22T19:04:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.171+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14400bc10 con 0x7fd150074d60 2024-03-22T19:04:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.171+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1440039d0 con 0x7fd150074d60 2024-03-22T19:04:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.171+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd150198490 con 0x7fd150074d60 2024-03-22T19:04:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.171+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd150198930 con 0x7fd150074d60 2024-03-22T19:04:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.172+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7fd144004390 con 0x7fd150074d60 2024-03-22T19:04:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.172+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd150191630 con 0x7fd150074d60 2024-03-22T19:04:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.172+0000 7fd14d600700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd1380385c0 0x7fd13803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.173+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd14404c280 con 0x7fd150074d60 2024-03-22T19:04:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.173+0000 7fd14fe00700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd1380385c0 0x7fd13803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.173+0000 7fd14fe00700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd1380385c0 0x7fd13803aa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.178+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd144029360 con 0x7fd150074d60 2024-03-22T19:04:02.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.403+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1500611f0 con 0x7fd1380385c0 2024-03-22T19:04:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.407+0000 7fd14d600700 1 -- 172.21.15.86:0/3204929737 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd1500611f0 con 0x7fd1380385c0 2024-03-22T19:04:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd1380385c0 msgr2=0x7fd13803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd1380385c0 0x7fd13803aa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 msgr2=0x7fd150197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd150197d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd144005950 tx=0x7fd1440057c0 comp rx=0 tx=0).stop 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 shutdown_connections 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd1380385c0 0x7fd13803aa80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 --2- 172.21.15.86:0/3204929737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150074d60 0x7fd150197d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 >> 172.21.15.86:0/3204929737 conn(0x7fd1500fbcb0 msgr2=0x7fd1500fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 shutdown_connections 2024-03-22T19:04:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:02.410+0000 7fd157c00700 1 -- 172.21.15.86:0/3204929737 wait complete. 2024-03-22T19:04:02.841 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi131 2024-03-22T19:04:02.842 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:04:02.842 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.conf 2024-03-22T19:04:02.862 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:04:02.862 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:02.922 INFO:tasks.cephadm:Adding host smithi131 to orchestrator... 2024-03-22T19:04:02.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch host add smithi131 2024-03-22T19:04:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:03.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:03 smithi086 ceph-mon[38032]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:03.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:03.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:03.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:03.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:03.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:03 smithi086 ceph-mon[38032]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-22T19:04:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 -- 172.21.15.86:0/2489395591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 msgr2=0x7fde80101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 --2- 172.21.15.86:0/2489395591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80101c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fde74009910 tx=0x7fde74009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 -- 172.21.15.86:0/2489395591 shutdown_connections 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 --2- 172.21.15.86:0/2489395591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80101c30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 -- 172.21.15.86:0/2489395591 >> 172.21.15.86:0/2489395591 conn(0x7fde800fb080 msgr2=0x7fde800fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 -- 172.21.15.86:0/2489395591 shutdown_connections 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.498+0000 7fde88000700 1 -- 172.21.15.86:0/2489395591 wait complete. 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.499+0000 7fde88000700 1 Processor -- start 2024-03-22T19:04:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.499+0000 7fde88000700 1 -- start start 2024-03-22T19:04:04.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde88000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:04.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde88000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde80196500 con 0x7fde80101850 2024-03-22T19:04:04.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde85600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde85600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55046/0 (socket says 172.21.15.86:55046) 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde85600700 1 -- 172.21.15.86:0/208820267 learned_addr learned my addr 172.21.15.86:0/208820267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde85600700 1 -- 172.21.15.86:0/208820267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740095f0 con 0x7fde80101850 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.500+0000 7fde85600700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80195fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fde74006010 tx=0x7fde74004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.501+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde7401d070 con 0x7fde80101850 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.501+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80196700 con 0x7fde80101850 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.501+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7400bc10 con 0x7fde80101850 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.501+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde740038f0 con 0x7fde80101850 2024-03-22T19:04:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.501+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80196b20 con 0x7fde80101850 2024-03-22T19:04:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.502+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7fde74003b60 con 0x7fde80101850 2024-03-22T19:04:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.502+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde800693b0 con 0x7fde80101850 2024-03-22T19:04:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.502+0000 7fde7e000700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde68038570 0x7fde6803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.502+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fde7404c1d0 con 0x7fde80101850 2024-03-22T19:04:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.503+0000 7fde84c00700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde68038570 0x7fde6803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.503+0000 7fde84c00700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde68038570 0x7fde6803aa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006fd0 tx=0x7fde6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.507+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fde74029540 con 0x7fde80101850 2024-03-22T19:04:04.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:04 smithi086 ceph-mon[38032]: Updating smithi086:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:04.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:04 smithi086 ceph-mon[38032]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:04.720+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}) v1 -- 0x7fde80105d30 con 0x7fde68038570 2024-03-22T19:04:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:05 smithi086 ceph-mon[38032]: Updating smithi086:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T19:04:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:06.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.887+0000 7fde7e000700 1 -- 172.21.15.86:0/208820267 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fde80105d30 con 0x7fde68038570 2024-03-22T19:04:06.887 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi131' with addr '172.21.15.131' 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.889+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde68038570 msgr2=0x7fde6803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.889+0000 7fde88000700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde68038570 0x7fde6803aa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006fd0 tx=0x7fde6c006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.889+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 msgr2=0x7fde80195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.889+0000 7fde88000700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80195fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fde74006010 tx=0x7fde74004ff0 comp rx=0 tx=0).stop 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.890+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 shutdown_connections 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.890+0000 7fde88000700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde68038570 0x7fde6803aa30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.890+0000 7fde88000700 1 --2- 172.21.15.86:0/208820267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde80101850 0x7fde80195fc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.890+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 >> 172.21.15.86:0/208820267 conn(0x7fde800fb080 msgr2=0x7fde80104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:06.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.890+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 shutdown_connections 2024-03-22T19:04:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:06.890+0000 7fde88000700 1 -- 172.21.15.86:0/208820267 wait complete. 2024-03-22T19:04:06.912 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:06 smithi086 ceph-mon[38032]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:06.912 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:06 smithi086 ceph-mon[38032]: Deploying cephadm binary to smithi131 2024-03-22T19:04:07.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-22T19:04:07.647 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:08.177 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:08.177 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:07 smithi086 ceph-mon[38032]: Added host smithi131 2024-03-22T19:04:08.177 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:08.177 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:08.177 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 -- 172.21.15.86:0/1348319392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 msgr2=0x7f3214106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 --2- 172.21.15.86:0/1348319392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214106a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3204009910 tx=0x7f3204009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 -- 172.21.15.86:0/1348319392 shutdown_connections 2024-03-22T19:04:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 --2- 172.21.15.86:0/1348319392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214106a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 -- 172.21.15.86:0/1348319392 >> 172.21.15.86:0/1348319392 conn(0x7f32140fc190 msgr2=0x7f32140fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 -- 172.21.15.86:0/1348319392 shutdown_connections 2024-03-22T19:04:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.824+0000 7f3219a00700 1 -- 172.21.15.86:0/1348319392 wait complete. 2024-03-22T19:04:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.825+0000 7f3219a00700 1 Processor -- start 2024-03-22T19:04:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.825+0000 7f3219a00700 1 -- start start 2024-03-22T19:04:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3219a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3219a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214198680 con 0x7f3214106680 2024-03-22T19:04:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3212a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3212a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40640/0 (socket says 172.21.15.86:40640) 2024-03-22T19:04:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3212a00700 1 -- 172.21.15.86:0/697361069 learned_addr learned my addr 172.21.15.86:0/697361069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3212a00700 1 -- 172.21.15.86:0/697361069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040095f0 con 0x7f3214106680 2024-03-22T19:04:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.826+0000 7f3212a00700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214198140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f3204004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.827+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f320401d070 con 0x7f3214106680 2024-03-22T19:04:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.827+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320400bbb0 con 0x7f3214106680 2024-03-22T19:04:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.827+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3204003810 con 0x7f3214106680 2024-03-22T19:04:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.827+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32141988e0 con 0x7f3214106680 2024-03-22T19:04:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.827+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3214198d60 con 0x7f3214106680 2024-03-22T19:04:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.828+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f320400bd20 con 0x7f3214106680 2024-03-22T19:04:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.828+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3214101dc0 con 0x7f3214106680 2024-03-22T19:04:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.828+0000 7f320b400700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31f80385c0 0x7f31f803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.828+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f320404c0f0 con 0x7f3214106680 2024-03-22T19:04:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.829+0000 7f3212000700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31f80385c0 0x7f31f803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.829+0000 7f3212000700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31f80385c0 0x7f31f803aa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3200006fd0 tx=0x7f3200006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:08.833+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3204029360 con 0x7f3214106680 2024-03-22T19:04:09.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.042+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3214108980 con 0x7f31f80385c0 2024-03-22T19:04:09.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.043+0000 7f320b400700 1 -- 172.21.15.86:0/697361069 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f3214108980 con 0x7f31f80385c0 2024-03-22T19:04:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:04:09.043 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}] 2024-03-22T19:04:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31f80385c0 msgr2=0x7f31f803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31f80385c0 0x7f31f803aa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3200006fd0 tx=0x7f3200006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 msgr2=0x7f3214198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214198140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f3204004ff0 comp rx=0 tx=0).stop 2024-03-22T19:04:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 shutdown_connections 2024-03-22T19:04:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31f80385c0 0x7f31f803aa80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 --2- 172.21.15.86:0/697361069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3214106680 0x7f3214198140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.045+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 >> 172.21.15.86:0/697361069 conn(0x7f32140fc190 msgr2=0x7f3214108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.046+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 shutdown_connections 2024-03-22T19:04:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:09.046+0000 7f3219a00700 1 -- 172.21.15.86:0/697361069 wait complete. 2024-03-22T19:04:09.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:09.422 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-03-22T19:04:09.422 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:04:09.422 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-03-22T19:04:09.443 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:04:09.443 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:09.503 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-03-22T19:04:09.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch host add smithi167 2024-03-22T19:04:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:10.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:10 smithi086 ceph-mon[38032]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:04:11.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 -- 172.21.15.86:0/1110317605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 msgr2=0x7ff540106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 --2- 172.21.15.86:0/1110317605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff540106a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff534009910 tx=0x7ff534009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 -- 172.21.15.86:0/1110317605 shutdown_connections 2024-03-22T19:04:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 --2- 172.21.15.86:0/1110317605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff540106a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 -- 172.21.15.86:0/1110317605 >> 172.21.15.86:0/1110317605 conn(0x7ff5400fc190 msgr2=0x7ff5400fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 -- 172.21.15.86:0/1110317605 shutdown_connections 2024-03-22T19:04:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.092+0000 7ff548000700 1 -- 172.21.15.86:0/1110317605 wait complete. 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.093+0000 7ff548000700 1 Processor -- start 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.093+0000 7ff548000700 1 -- start start 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.094+0000 7ff548000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff5401981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.094+0000 7ff548000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5401986e0 con 0x7ff540106680 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.094+0000 7ff545600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff5401981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.094+0000 7ff545600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff5401981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40652/0 (socket says 172.21.15.86:40652) 2024-03-22T19:04:11.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.094+0000 7ff545600700 1 -- 172.21.15.86:0/4014382887 learned_addr learned my addr 172.21.15.86:0/4014382887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.094+0000 7ff545600700 1 -- 172.21.15.86:0/4014382887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340095f0 con 0x7ff540106680 2024-03-22T19:04:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.095+0000 7ff545600700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff5401981a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff534006010 tx=0x7ff534004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.095+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff53401d070 con 0x7ff540106680 2024-03-22T19:04:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.095+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53400bbb0 con 0x7ff540106680 2024-03-22T19:04:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.095+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540198940 con 0x7ff540106680 2024-03-22T19:04:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.095+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff534003810 con 0x7ff540106680 2024-03-22T19:04:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.095+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540198dc0 con 0x7ff540106680 2024-03-22T19:04:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.096+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540101dc0 con 0x7ff540106680 2024-03-22T19:04:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.096+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7ff53400bd20 con 0x7ff540106680 2024-03-22T19:04:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.097+0000 7ff53e000700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5280385c0 0x7ff52803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.097+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff53404c150 con 0x7ff540106680 2024-03-22T19:04:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.097+0000 7ff544c00700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5280385c0 0x7ff52803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:11.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.098+0000 7ff544c00700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5280385c0 0x7ff52803aa80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:11.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.102+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff53402aba0 con 0x7ff540106680 2024-03-22T19:04:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:11.314+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) v1 -- 0x7ff540199550 con 0x7ff5280385c0 2024-03-22T19:04:12.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:12.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:12.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: Updating smithi131:/etc/ceph/ceph.conf 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:12.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:13.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:12 smithi086 ceph-mon[38032]: Updating smithi131:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:13.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:12 smithi086 ceph-mon[38032]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:13.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:12 smithi086 ceph-mon[38032]: Updating smithi131:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T19:04:13.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:12 smithi086 ceph-mon[38032]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.521+0000 7ff53e000700 1 -- 172.21.15.86:0/4014382887 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff540199550 con 0x7ff5280385c0 2024-03-22T19:04:13.523 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-03-22T19:04:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5280385c0 msgr2=0x7ff52803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5280385c0 0x7ff52803aa80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 msgr2=0x7ff5401981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff5401981a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff534006010 tx=0x7ff534004ff0 comp rx=0 tx=0).stop 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 shutdown_connections 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5280385c0 0x7ff52803aa80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 --2- 172.21.15.86:0/4014382887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff540106680 0x7ff5401981a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 >> 172.21.15.86:0/4014382887 conn(0x7ff5400fc190 msgr2=0x7ff540108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 shutdown_connections 2024-03-22T19:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:13.523+0000 7ff548000700 1 -- 172.21.15.86:0/4014382887 wait complete. 2024-03-22T19:04:13.884 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:13 smithi086 ceph-mon[38032]: Deploying cephadm binary to smithi167 2024-03-22T19:04:13.884 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:13 smithi086 ceph-mon[38032]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:13.884 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:13.884 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:13.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-22T19:04:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:14.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:14 smithi086 ceph-mon[38032]: Added host smithi167 2024-03-22T19:04:14.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:14.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 -- 172.21.15.86:0/1745924930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 msgr2=0x7f7fd0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 --2- 172.21.15.86:0/1745924930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd0106a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009910 tx=0x7f7fc4009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 -- 172.21.15.86:0/1745924930 shutdown_connections 2024-03-22T19:04:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 --2- 172.21.15.86:0/1745924930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd0106a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 -- 172.21.15.86:0/1745924930 >> 172.21.15.86:0/1745924930 conn(0x7f7fd00fc190 msgr2=0x7f7fd00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 -- 172.21.15.86:0/1745924930 shutdown_connections 2024-03-22T19:04:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.431+0000 7f7fd8400700 1 -- 172.21.15.86:0/1745924930 wait complete. 2024-03-22T19:04:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.432+0000 7f7fd8400700 1 Processor -- start 2024-03-22T19:04:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.432+0000 7f7fd8400700 1 -- start start 2024-03-22T19:04:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.432+0000 7f7fd8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.432+0000 7f7fd8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd01986e0 con 0x7f7fd0106680 2024-03-22T19:04:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.433+0000 7f7fd5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.433+0000 7f7fd5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd01981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40668/0 (socket says 172.21.15.86:40668) 2024-03-22T19:04:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.433+0000 7f7fd5a00700 1 -- 172.21.15.86:0/1990526231 learned_addr learned my addr 172.21.15.86:0/1990526231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.433+0000 7f7fd5a00700 1 -- 172.21.15.86:0/1990526231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40095f0 con 0x7f7fd0106680 2024-03-22T19:04:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.433+0000 7f7fd5a00700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd01981a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006010 tx=0x7f7fc4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.434+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fc401d070 con 0x7f7fd0106680 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.434+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc400bbb0 con 0x7f7fd0106680 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.434+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fc4003810 con 0x7f7fd0106680 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.434+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd0198940 con 0x7f7fd0106680 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.434+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd0198dc0 con 0x7f7fd0106680 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.435+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd0101dc0 con 0x7f7fd0106680 2024-03-22T19:04:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.435+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f7fc400b6f0 con 0x7f7fd0106680 2024-03-22T19:04:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.435+0000 7f7fce000700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7fb80385c0 0x7f7fb803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.435+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7fc404c170 con 0x7f7fd0106680 2024-03-22T19:04:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.436+0000 7f7fd5000700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7fb80385c0 0x7f7fb803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.436+0000 7f7fd5000700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7fb80385c0 0x7f7fb803aa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc006fd0 tx=0x7f7fbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.440+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7fc4029540 con 0x7f7fd0106680 2024-03-22T19:04:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.647+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fd0199550 con 0x7f7fb80385c0 2024-03-22T19:04:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.648+0000 7f7fce000700 1 -- 172.21.15.86:0/1990526231 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f7fd0199550 con 0x7f7fb80385c0 2024-03-22T19:04:15.649 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:04:15.649 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.650+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7fb80385c0 msgr2=0x7f7fb803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.650+0000 7f7fd8400700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7fb80385c0 0x7f7fb803aa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc006fd0 tx=0x7f7fbc006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.650+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 msgr2=0x7f7fd01981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.650+0000 7f7fd8400700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd01981a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006010 tx=0x7f7fc4004ff0 comp rx=0 tx=0).stop 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.651+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 shutdown_connections 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.651+0000 7f7fd8400700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7fb80385c0 0x7f7fb803aa80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.651+0000 7f7fd8400700 1 --2- 172.21.15.86:0/1990526231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd0106680 0x7f7fd01981a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.651+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 >> 172.21.15.86:0/1990526231 conn(0x7f7fd00fc190 msgr2=0x7f7fd0108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.651+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 shutdown_connections 2024-03-22T19:04:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:15.651+0000 7f7fd8400700 1 -- 172.21.15.86:0/1990526231 wait complete. 2024-03-22T19:04:15.850 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:15 smithi086 ceph-mon[38032]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:15.850 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:16.082 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-22T19:04:16.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-22T19:04:16.424 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.602+0000 7f4a62400700 1 -- 172.21.15.86:0/902222491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 msgr2=0x7f4a5c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.602+0000 7f4a62400700 1 --2- 172.21.15.86:0/902222491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c106a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009910 tx=0x7f4a4c009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.602+0000 7f4a62400700 1 -- 172.21.15.86:0/902222491 shutdown_connections 2024-03-22T19:04:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.602+0000 7f4a62400700 1 --2- 172.21.15.86:0/902222491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c106a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.602+0000 7f4a62400700 1 -- 172.21.15.86:0/902222491 >> 172.21.15.86:0/902222491 conn(0x7f4a5c0fc190 msgr2=0x7f4a5c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.603+0000 7f4a62400700 1 -- 172.21.15.86:0/902222491 shutdown_connections 2024-03-22T19:04:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.603+0000 7f4a62400700 1 -- 172.21.15.86:0/902222491 wait complete. 2024-03-22T19:04:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a62400700 1 Processor -- start 2024-03-22T19:04:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a62400700 1 -- start start 2024-03-22T19:04:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a62400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a62400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c198660 con 0x7f4a5c106680 2024-03-22T19:04:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40674/0 (socket says 172.21.15.86:40674) 2024-03-22T19:04:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.604+0000 7f4a5b400700 1 -- 172.21.15.86:0/2535729966 learned_addr learned my addr 172.21.15.86:0/2535729966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a5b400700 1 -- 172.21.15.86:0/2535729966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0095f0 con 0x7f4a5c106680 2024-03-22T19:04:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a5b400700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c198120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c006010 tx=0x7f4a4c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a4c01d070 con 0x7f4a5c106680 2024-03-22T19:04:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c00bbb0 con 0x7f4a5c106680 2024-03-22T19:04:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a4c003810 con 0x7f4a5c106680 2024-03-22T19:04:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c1988c0 con 0x7f4a5c106680 2024-03-22T19:04:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.605+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c198d40 con 0x7f4a5c106680 2024-03-22T19:04:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.607+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c101dc0 con 0x7f4a5c106680 2024-03-22T19:04:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.607+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f4a4c00bd20 con 0x7f4a5c106680 2024-03-22T19:04:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.607+0000 7f4a53e00700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4a400385c0 0x7f4a4003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.607+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a4c04c190 con 0x7f4a5c106680 2024-03-22T19:04:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.608+0000 7f4a5aa00700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4a400385c0 0x7f4a4003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.608+0000 7f4a5aa00700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4a400385c0 0x7f4a4003aa80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006fd0 tx=0x7f4a48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.612+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4a4c02aba0 con 0x7f4a5c106680 2024-03-22T19:04:17.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:04:17.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:17.825+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4a5c0623e0 con 0x7f4a5c106680 2024-03-22T19:04:18.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.569+0000 7f4a53e00700 1 -- 172.21.15.86:0/2535729966 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4a4c026020 con 0x7f4a5c106680 2024-03-22T19:04:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4a400385c0 msgr2=0x7f4a4003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4a400385c0 0x7f4a4003aa80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006fd0 tx=0x7f4a48006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 msgr2=0x7f4a5c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c198120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c006010 tx=0x7f4a4c004ff0 comp rx=0 tx=0).stop 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 shutdown_connections 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4a400385c0 0x7f4a4003aa80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 --2- 172.21.15.86:0/2535729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a5c106680 0x7f4a5c198120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 >> 172.21.15.86:0/2535729966 conn(0x7f4a5c0fc190 msgr2=0x7f4a5c108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 shutdown_connections 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:18.572+0000 7f4a62400700 1 -- 172.21.15.86:0/2535729966 wait complete. 2024-03-22T19:04:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-03-22T19:04:18.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:18 smithi086 ceph-mon[38032]: Updating smithi167:/etc/ceph/ceph.conf 2024-03-22T19:04:18.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:18 smithi086 ceph-mon[38032]: Updating smithi167:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:18.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:18 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2535729966' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-22T19:04:18.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:18.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:18.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:18.949 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-03-22T19:04:18.949 INFO:tasks.cephadm:Adding mon.b on smithi131 2024-03-22T19:04:18.949 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-03-22T19:04:18.949 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi131:172.21.15.131=b;smithi167:172.21.15.167=c' 2024-03-22T19:04:19.211 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:19 smithi086 ceph-mon[38032]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:19 smithi086 ceph-mon[38032]: Updating smithi167:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T19:04:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:19 smithi086 ceph-mon[38032]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:19 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2535729966' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-22T19:04:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:19 smithi086 ceph-mon[38032]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:20.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 -- 172.21.15.167:0/3831065099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 msgr2=0x7f57d0105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 --2- 172.21.15.167:0/3831065099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0105a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009910 tx=0x7f57c4009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 -- 172.21.15.167:0/3831065099 shutdown_connections 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 --2- 172.21.15.167:0/3831065099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0105a40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 -- 172.21.15.167:0/3831065099 >> 172.21.15.167:0/3831065099 conn(0x7f57d0075110 msgr2=0x7f57d0075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 -- 172.21.15.167:0/3831065099 shutdown_connections 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.155+0000 7f57d6f8d700 1 -- 172.21.15.167:0/3831065099 wait complete. 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d6f8d700 1 Processor -- start 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d6f8d700 1 -- start start 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d6f8d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:20.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d6f8d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0068e60 con 0x7f57d0105660 2024-03-22T19:04:20.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d4d29700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d4d29700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.167:56292/0 (socket says 172.21.15.167:56292) 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.156+0000 7f57d4d29700 1 -- 172.21.15.167:0/886434019 learned_addr learned my addr 172.21.15.167:0/886434019 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.157+0000 7f57d4d29700 1 -- 172.21.15.167:0/886434019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40095f0 con 0x7f57d0105660 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.157+0000 7f57d4d29700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0196cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006010 tx=0x7f57c4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.157+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c401d070 con 0x7f57d0105660 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.157+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c400bc10 con 0x7f57d0105660 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.157+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d0197250 con 0x7f57d0105660 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.157+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c40038a0 con 0x7f57d0105660 2024-03-22T19:04:20.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.158+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d0197750 con 0x7f57d0105660 2024-03-22T19:04:20.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.159+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d00fee50 con 0x7f57d0105660 2024-03-22T19:04:20.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.159+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f57c400b750 con 0x7f57d0105660 2024-03-22T19:04:20.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.159+0000 7f57cdffb700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57c00385c0 0x7f57c003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:20.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.159+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f57c404c150 con 0x7f57d0105660 2024-03-22T19:04:20.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.160+0000 7f57cffff700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57c00385c0 0x7f57c003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:20.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.160+0000 7f57cffff700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57c00385c0 0x7f57c003aa80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:20.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.164+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57c400bd80 con 0x7f57d0105660 2024-03-22T19:04:20.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.367+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi131:172.21.15.131=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f57d0068540 con 0x7f57c00385c0 2024-03-22T19:04:20.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.376+0000 7f57cdffb700 1 -- 172.21.15.167:0/886434019 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f57d0068540 con 0x7f57c00385c0 2024-03-22T19:04:20.377 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57c00385c0 msgr2=0x7f57c003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57c00385c0 0x7f57c003aa80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 msgr2=0x7f57d0196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0196cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006010 tx=0x7f57c4004ff0 comp rx=0 tx=0).stop 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 shutdown_connections 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57c00385c0 0x7f57c003aa80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 --2- 172.21.15.167:0/886434019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0105660 0x7f57d0196cb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 >> 172.21.15.167:0/886434019 conn(0x7f57d0075110 msgr2=0x7f57d00fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 shutdown_connections 2024-03-22T19:04:20.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:20.377+0000 7f57d6f8d700 1 -- 172.21.15.167:0/886434019 wait complete. 2024-03-22T19:04:20.763 DEBUG:teuthology.orchestra.run.smithi131:mon.b> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.b.service 2024-03-22T19:04:20.766 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.c.service 2024-03-22T19:04:20.769 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-22T19:04:20.769 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T19:04:20.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:04:20.803 INFO:journalctl@ceph.mon.b.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:04:21.196 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:21.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:21.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi131:172.21.15.131=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi131:172.21.15.131=b;smithi167:172.21.15.167=c;count:3 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:21 smithi086 ceph-mon[38032]: Deploying daemon mon.c on smithi167 2024-03-22T19:04:22.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 -- 172.21.15.167:0/507316624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 msgr2=0x7fad48106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 --2- 172.21.15.167:0/507316624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48106a80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009910 tx=0x7fad3c009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 -- 172.21.15.167:0/507316624 shutdown_connections 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 --2- 172.21.15.167:0/507316624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48106a80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 -- 172.21.15.167:0/507316624 >> 172.21.15.167:0/507316624 conn(0x7fad480fc1b0 msgr2=0x7fad480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 -- 172.21.15.167:0/507316624 shutdown_connections 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 -- 172.21.15.167:0/507316624 wait complete. 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 Processor -- start 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.865+0000 7fad4e4b4700 1 -- start start 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.866+0000 7fad4e4b4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.866+0000 7fad4e4b4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad481964b0 con 0x7fad481066a0 2024-03-22T19:04:22.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.866+0000 7fad47fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.866+0000 7fad47fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.167:56320/0 (socket says 172.21.15.167:56320) 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.866+0000 7fad47fff700 1 -- 172.21.15.167:0/3385214434 learned_addr learned my addr 172.21.15.167:0/3385214434 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad47fff700 1 -- 172.21.15.167:0/3385214434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0095f0 con 0x7fad481066a0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad47fff700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48195f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad3c004f40 tx=0x7fad3c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad3c01d070 con 0x7fad481066a0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad48196710 con 0x7fad481066a0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad3c00bbb0 con 0x7fad481066a0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad3c003810 con 0x7fad481066a0 2024-03-22T19:04:22.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.867+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad48196b90 con 0x7fad481066a0 2024-03-22T19:04:22.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.868+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad48101de0 con 0x7fad481066a0 2024-03-22T19:04:22.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.868+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7fad3c00b6f0 con 0x7fad481066a0 2024-03-22T19:04:22.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.868+0000 7fad457fa700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fad30038570 0x7fad3003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:22.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.868+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad3c04c100 con 0x7fad481066a0 2024-03-22T19:04:22.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.869+0000 7fad477fe700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fad30038570 0x7fad3003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:22.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.870+0000 7fad477fe700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fad30038570 0x7fad3003aa30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad38006fd0 tx=0x7fad38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:22.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:22.874+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad3c029540 con 0x7fad481066a0 2024-03-22T19:04:23.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.169+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fad4819a810 con 0x7fad481066a0 2024-03-22T19:04:23.173 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T19:04:23.173 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","modified":"2024-03-22T19:02:52.431583Z","created":"2024-03-22T19:02:52.431583Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T19:04:23.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.170+0000 7fad457fa700 1 -- 172.21.15.167:0/3385214434 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fad3c026070 con 0x7fad481066a0 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fad30038570 msgr2=0x7fad3003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fad30038570 0x7fad3003aa30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad38006fd0 tx=0x7fad38006e40 comp rx=0 tx=0).stop 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 msgr2=0x7fad48195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48195f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad3c004f40 tx=0x7fad3c005020 comp rx=0 tx=0).stop 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 shutdown_connections 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fad30038570 0x7fad3003aa30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 --2- 172.21.15.167:0/3385214434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad481066a0 0x7fad48195f70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 >> 172.21.15.167:0/3385214434 conn(0x7fad480fc1b0 msgr2=0x7fad48108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 shutdown_connections 2024-03-22T19:04:23.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:23.173+0000 7fad4e4b4700 1 -- 172.21.15.167:0/3385214434 wait complete. 2024-03-22T19:04:23.176 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-03-22T19:04:23.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:23 smithi086 ceph-mon[38032]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:23.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:23 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/3385214434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T19:04:23.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:23 smithi167 ceph-mon[47728]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-22T19:04:24.887 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-22T19:04:24.888 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T19:04:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: mon.a calling monitor election 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: mon.c calling monitor election 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:28.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: monmap e2: 2 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: fsmap 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: mgrmap e13: x(active, since 38s) 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: overall HEALTH_OK 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:28.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:29.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:04:29.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:29.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: mon.a calling monitor election 2024-03-22T19:04:29.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:29.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:29.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:29.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: mon.c calling monitor election 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: monmap e2: 2 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: fsmap 2024-03-22T19:04:29.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:29.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: mgrmap e13: x(active, since 38s) 2024-03-22T19:04:29.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: overall HEALTH_OK 2024-03-22T19:04:29.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:29.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:29.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:29.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:29.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:29.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.102+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/3628605768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 msgr2=0x7fdd20007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:29.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.102+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/3628605768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd20007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd380ff5e0 tx=0x7fdd2c00feb0 comp rx=0 tx=0).stop 2024-03-22T19:04:29.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.102+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/3628605768 shutdown_connections 2024-03-22T19:04:29.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.102+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/3628605768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd20007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:29.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.102+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/3628605768 >> 172.21.15.167:0/3628605768 conn(0x7fdd380fb1c0 msgr2=0x7fdd380fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:29.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.102+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/3628605768 shutdown_connections 2024-03-22T19:04:29.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.103+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/3628605768 wait complete. 2024-03-22T19:04:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.104+0000 7fdd3d0bc700 1 Processor -- start 2024-03-22T19:04:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.104+0000 7fdd3d0bc700 1 -- start start 2024-03-22T19:04:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.104+0000 7fdd3d0bc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd38193ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.104+0000 7fdd3d0bc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.104+0000 7fdd3d0bc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381949b0 con 0x7fdd200057d0 2024-03-22T19:04:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.104+0000 7fdd3d0bc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38194b20 con 0x7fdd38194420 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40898/0 (socket says 172.21.15.167:40898) 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 -- 172.21.15.167:0/770325651 learned_addr learned my addr 172.21.15.167:0/770325651 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 -- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 msgr2=0x7fdd381987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 -- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 msgr2=0x7fdd381987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd36d9d700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd38193ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:29.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd3659c700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd36d9d700 1 -- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 msgr2=0x7fdd381987d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd36d9d700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd36d9d700 1 -- 172.21.15.167:0/770325651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c00f900 con 0x7fdd200057d0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.105+0000 7fdd36d9d700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd38193ee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00f800 tx=0x7fdd2c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.106+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fdd2c013960 con 0x7fdd200057d0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.106+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2c004030 con 0x7fdd200057d0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.106+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd38198d70 con 0x7fdd200057d0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.106+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fdd2c003ac0 con 0x7fdd200057d0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.106+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd381992c0 con 0x7fdd200057d0 2024-03-22T19:04:29.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.107+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7fdd2c011540 con 0x7fdd200057d0 2024-03-22T19:04:29.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.107+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3818dc90 con 0x7fdd200057d0 2024-03-22T19:04:29.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.107+0000 7fdd1ffff700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd240383b0 0x7fdd2403a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:29.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.107+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdd2c000e40 con 0x7fdd200057d0 2024-03-22T19:04:29.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.107+0000 7fdd3659c700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd240383b0 0x7fdd2403a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:29.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.108+0000 7fdd3659c700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd240383b0 0x7fdd2403a870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009180 tx=0x7fdd28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:29.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.112+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdd2c01a980 con 0x7fdd200057d0 2024-03-22T19:04:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.381+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdd380623e0 con 0x7fdd200057d0 2024-03-22T19:04:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.382+0000 7fdd1ffff700 1 -- 172.21.15.167:0/770325651 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fdd2c01a390 con 0x7fdd200057d0 2024-03-22T19:04:29.383 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T19:04:29.383 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","modified":"2024-03-22T19:04:23.470704Z","created":"2024-03-22T19:02:52.431583Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-22T19:04:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd240383b0 msgr2=0x7fdd2403a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd240383b0 0x7fdd2403a870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009180 tx=0x7fdd28008040 comp rx=0 tx=0).stop 2024-03-22T19:04:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 msgr2=0x7fdd38193ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd38193ee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00f800 tx=0x7fdd2c004f80 comp rx=0 tx=0).stop 2024-03-22T19:04:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 shutdown_connections 2024-03-22T19:04:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd240383b0 0x7fdd2403a870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd200057d0 0x7fdd38193ee0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 --2- 172.21.15.167:0/770325651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd38194420 0x7fdd381987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 >> 172.21.15.167:0/770325651 conn(0x7fdd380fb1c0 msgr2=0x7fdd380fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 shutdown_connections 2024-03-22T19:04:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:29.384+0000 7fdd3d0bc700 1 -- 172.21.15.167:0/770325651 wait complete. 2024-03-22T19:04:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-03-22T19:04:29.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:29 smithi167 ceph-mon[47728]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:29.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:29 smithi167 ceph-mon[47728]: Deploying daemon mon.b on smithi131 2024-03-22T19:04:29.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:29 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/770325651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T19:04:29.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:30.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:29 smithi086 ceph-mon[38032]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:30.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:29 smithi086 ceph-mon[38032]: Deploying daemon mon.b on smithi131 2024-03-22T19:04:30.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:29 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/770325651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T19:04:30.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:30.772 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-22T19:04:30.773 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T19:04:31.001 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:04:31.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:31 smithi131 bash[78522]: d1576d41fad957603b89781fb37bec9d06820c89324b0879f891f2c48a21ba2e 2024-03-22T19:04:31.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:31 smithi131 systemd[1]: Started Ceph mon.b for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:04:36.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 -- 172.21.15.167:0/2325531818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc501056e0 msgr2=0x7fcc38007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:36.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2325531818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc501056e0 0x7fcc38007820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005fa0 tx=0x7fcc4000ff70 comp rx=0 tx=0).stop 2024-03-22T19:04:36.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 -- 172.21.15.167:0/2325531818 shutdown_connections 2024-03-22T19:04:36.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2325531818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc501056e0 0x7fcc38007820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 -- 172.21.15.167:0/2325531818 >> 172.21.15.167:0/2325531818 conn(0x7fcc500fb250 msgr2=0x7fcc500fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 -- 172.21.15.167:0/2325531818 shutdown_connections 2024-03-22T19:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.336+0000 7fcc556b1700 1 -- 172.21.15.167:0/2325531818 wait complete. 2024-03-22T19:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 Processor -- start 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 -- start start 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc50194460 0x7fcc501987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 0x7fcc501991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50194b50 con 0x7fcc50194460 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a0a70 con 0x7fcc50198d00 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.337+0000 7fcc556b1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a0bb0 con 0x7fcc501056e0 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40906/0 (socket says 172.21.15.167:40906) 2024-03-22T19:04:36.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 -- 172.21.15.167:0/2225258765 learned_addr learned my addr 172.21.15.167:0/2225258765 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4f7fe700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 0x7fcc501991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 msgr2=0x7fcc50193f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 msgr2=0x7fcc50193f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4effd700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc50194460 0x7fcc501987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4f7fe700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 msgr2=0x7fcc501991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4f7fe700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 msgr2=0x7fcc501991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4f7fe700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 0x7fcc501991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4f7fe700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 0x7fcc501991b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 msgr2=0x7fcc50193f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 msgr2=0x7fcc501991b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:04:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 0x7fcc501991b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 -- 172.21.15.167:0/2225258765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4000f970 con 0x7fcc50194460 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.338+0000 7fcc4e7fc700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc50194460 0x7fcc501987c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b550 tx=0x7fcc4400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.339+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44004230 con 0x7fcc50194460 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.339+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc44004390 con 0x7fcc50194460 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.339+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc501a0e90 con 0x7fcc50194460 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.339+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44011af0 con 0x7fcc50194460 2024-03-22T19:04:36.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.339+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc501a1330 con 0x7fcc50194460 2024-03-22T19:04:36.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.340+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7fcc44011c50 con 0x7fcc50194460 2024-03-22T19:04:36.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.340+0000 7fcc37fff700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcc3c038a40 0x7fcc3c03af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:36.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.340+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc4404add0 con 0x7fcc50194460 2024-03-22T19:04:36.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.340+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5018dbe0 con 0x7fcc50194460 2024-03-22T19:04:36.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.340+0000 7fcc4effd700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcc3c038a40 0x7fcc3c03af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:36.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.341+0000 7fcc4effd700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcc3c038a40 0x7fcc3c03af00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005850 tx=0x7fcc40005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:36.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.346+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcc4401a400 con 0x7fcc50194460 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: mon.c calling monitor election 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: mon.a calling monitor election 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:36.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: fsmap 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: mgrmap e13: x(active, since 46s) 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: mon.b (rank 2) addr [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] is down (out of quorum) 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: mon.c calling monitor election 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: mon.a calling monitor election 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: fsmap 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: mgrmap e13: x(active, since 46s) 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-22T19:04:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: mon.b (rank 2) addr [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] is down (out of quorum) 2024-03-22T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:36.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.644+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc500623e0 con 0x7fcc50194460 2024-03-22T19:04:36.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.645+0000 7fcc37fff700 1 -- 172.21.15.167:0/2225258765 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fcc4403e030 con 0x7fcc50194460 2024-03-22T19:04:36.646 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-03-22T19:04:36.646 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","modified":"2024-03-22T19:04:31.076650Z","created":"2024-03-22T19:02:52.431583Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-22T19:04:36.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcc3c038a40 msgr2=0x7fcc3c03af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:36.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcc3c038a40 0x7fcc3c03af00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005850 tx=0x7fcc40005da0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc50194460 msgr2=0x7fcc501987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:36.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc50194460 0x7fcc501987c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b550 tx=0x7fcc4400b910 comp rx=0 tx=0).stop 2024-03-22T19:04:36.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 shutdown_connections 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcc3c038a40 0x7fcc3c03af00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc50198d00 0x7fcc501991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc501056e0 0x7fcc50193f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 --2- 172.21.15.167:0/2225258765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc50194460 0x7fcc501987c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 >> 172.21.15.167:0/2225258765 conn(0x7fcc500fb250 msgr2=0x7fcc500fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 shutdown_connections 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:36.647+0000 7fcc556b1700 1 -- 172.21.15.167:0/2225258765 wait complete. 2024-03-22T19:04:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-03-22T19:04:37.067 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-22T19:04:37.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-22T19:04:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:37 smithi167 ceph-mon[47728]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:37 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/2225258765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T19:04:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:37 smithi086 ceph-mon[38032]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:37 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/2225258765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T19:04:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:37.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:37.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.c calling monitor election 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.a calling monitor election 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:38.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: fsmap 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mgrmap e13: x(active, since 46s) 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.b (rank 2) addr [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] is down (out of quorum) 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/2225258765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T19:04:38.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:38.474 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.474 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.731+0000 7fa479800700 1 -- 172.21.15.86:0/2322157871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 msgr2=0x7fa474106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.731+0000 7fa479800700 1 --2- 172.21.15.86:0/2322157871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa474106a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa464009910 tx=0x7fa464009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.731+0000 7fa479800700 1 -- 172.21.15.86:0/2322157871 shutdown_connections 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.731+0000 7fa479800700 1 --2- 172.21.15.86:0/2322157871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa474106a80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.731+0000 7fa479800700 1 -- 172.21.15.86:0/2322157871 >> 172.21.15.86:0/2322157871 conn(0x7fa474075110 msgr2=0x7fa474075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.732+0000 7fa479800700 1 -- 172.21.15.86:0/2322157871 shutdown_connections 2024-03-22T19:04:38.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.732+0000 7fa479800700 1 -- 172.21.15.86:0/2322157871 wait complete. 2024-03-22T19:04:38.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.733+0000 7fa479800700 1 Processor -- start 2024-03-22T19:04:38.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.733+0000 7fa479800700 1 -- start start 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa479800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa4741960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa479800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4741965e0 0x7fa47419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa479800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47419aed0 0x7fa47419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa479800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474196cd0 con 0x7fa4741066a0 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa479800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a2c40 con 0x7fa4741965e0 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa479800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a2d80 con 0x7fa47419aed0 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa472a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa4741960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:38.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa472a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa4741960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39674/0 (socket says 172.21.15.86:39674) 2024-03-22T19:04:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa472a00700 1 -- 172.21.15.86:0/2228811949 learned_addr learned my addr 172.21.15.86:0/2228811949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:38.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa472000700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4741965e0 0x7fa47419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.734+0000 7fa473400700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47419aed0 0x7fa47419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa472a00700 1 -- 172.21.15.86:0/2228811949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47419aed0 msgr2=0x7fa47419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa472a00700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47419aed0 0x7fa47419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa472a00700 1 -- 172.21.15.86:0/2228811949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4741965e0 msgr2=0x7fa47419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa472a00700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4741965e0 0x7fa47419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa472a00700 1 -- 172.21.15.86:0/2228811949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640095f0 con 0x7fa4741066a0 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa472a00700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa4741960a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa464004cb0 tx=0x7fa464004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46401d070 con 0x7fa4741066a0 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa464005000 con 0x7fa4741066a0 2024-03-22T19:04:38.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4741a3060 con 0x7fa4741066a0 2024-03-22T19:04:38.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464022c90 con 0x7fa4741066a0 2024-03-22T19:04:38.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.735+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4741a35b0 con 0x7fa4741066a0 2024-03-22T19:04:38.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.736+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7fa46400b960 con 0x7fa4741066a0 2024-03-22T19:04:38.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.737+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa474101de0 con 0x7fa4741066a0 2024-03-22T19:04:38.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.737+0000 7fa46b400700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa458038a40 0x7fa45803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:38.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.737+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa46404c7f0 con 0x7fa4741066a0 2024-03-22T19:04:38.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.737+0000 7fa472000700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa458038a40 0x7fa45803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:38.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.738+0000 7fa472000700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa458038a40 0x7fa45803af00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa460009ad0 tx=0x7fa460009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:38.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.742+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa46400ebd0 con 0x7fa4741066a0 2024-03-22T19:04:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.957+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa4740623e0 con 0x7fa4741066a0 2024-03-22T19:04:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.957+0000 7fa46b400700 1 -- 172.21.15.86:0/2228811949 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fa46400ebd0 con 0x7fa4741066a0 2024-03-22T19:04:38.959 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:04:38.959 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-22T19:04:38.959 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:04:38.959 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.959+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa458038a40 msgr2=0x7fa45803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.959+0000 7fa479800700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa458038a40 0x7fa45803af00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa460009ad0 tx=0x7fa460009210 comp rx=0 tx=0).stop 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 msgr2=0x7fa4741960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa4741960a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa464004cb0 tx=0x7fa464004ce0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 shutdown_connections 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa458038a40 0x7fa45803af00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47419aed0 0x7fa47419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4741066a0 0x7fa4741960a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 --2- 172.21.15.86:0/2228811949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4741965e0 0x7fa47419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 >> 172.21.15.86:0/2228811949 conn(0x7fa474075110 msgr2=0x7fa4740fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 shutdown_connections 2024-03-22T19:04:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:38.960+0000 7fa479800700 1 -- 172.21.15.86:0/2228811949 wait complete. 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: mon.b calling monitor election 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: Updating smithi131:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: Updating smithi167:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: Updating smithi086:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: mon.b calling monitor election 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: mon.c calling monitor election 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: mon.a calling monitor election 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:39.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: fsmap 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: mgrmap e13: x(active, since 48s) 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: Cluster is now healthy 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: overall HEALTH_OK 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:39.172 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:39.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.b calling monitor election 2024-03-22T19:04:39.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Updating smithi131:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Updating smithi167:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Updating smithi086:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.b calling monitor election 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.c calling monitor election 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.a calling monitor election 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: fsmap 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:39.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: mgrmap e13: x(active, since 48s) 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: Cluster is now healthy 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: overall HEALTH_OK 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:39.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: mon.b calling monitor election 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: Updating smithi131:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: Updating smithi167:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: Updating smithi086:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: mon.b calling monitor election 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: mon.c calling monitor election 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: mon.a calling monitor election 2024-03-22T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: fsmap 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: mgrmap e13: x(active, since 48s) 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: Cluster is now healthy 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: overall HEALTH_OK 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:39.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:39.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:39.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:39.444 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-22T19:04:39.444 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:04:39.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T19:04:39.515 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:04:39.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:39.620 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:04:39.620 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T19:04:39.657 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:04:39.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:39.728 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:04:39.728 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T19:04:39.764 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:04:39.764 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T19:04:39.835 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-03-22T19:04:39.836 INFO:tasks.cephadm:Adding mgr.y on smithi131 2024-03-22T19:04:39.836 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-03-22T19:04:39.836 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi086=x;smithi131=y;smithi167=z' 2024-03-22T19:04:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:39 smithi131 ceph-mon[78559]: Reconfiguring mon.a (unknown last config time)... 2024-03-22T19:04:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:39 smithi131 ceph-mon[78559]: Reconfiguring daemon mon.a on smithi086 2024-03-22T19:04:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:39 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2228811949' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:39 smithi086 ceph-mon[38032]: Reconfiguring mon.a (unknown last config time)... 2024-03-22T19:04:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:39 smithi086 ceph-mon[38032]: Reconfiguring daemon mon.a on smithi086 2024-03-22T19:04:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:39 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2228811949' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:40.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:40.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:39 smithi167 ceph-mon[47728]: Reconfiguring mon.a (unknown last config time)... 2024-03-22T19:04:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:39 smithi167 ceph-mon[47728]: Reconfiguring daemon mon.a on smithi086 2024-03-22T19:04:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:39 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2228811949' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:04:40.092 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:04:41.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.115+0000 7f24e4945700 1 -- 172.21.15.167:0/4030274493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01012a0 msgr2=0x7f24e0103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:41.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.115+0000 7f24e4945700 1 --2- 172.21.15.167:0/4030274493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01012a0 0x7f24e0103690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009910 tx=0x7f24d0009c20 comp rx=0 tx=0).stop 2024-03-22T19:04:41.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.115+0000 7f24e4945700 1 -- 172.21.15.167:0/4030274493 shutdown_connections 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.115+0000 7f24e4945700 1 --2- 172.21.15.167:0/4030274493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01012a0 0x7f24e0103690 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.115+0000 7f24e4945700 1 -- 172.21.15.167:0/4030274493 >> 172.21.15.167:0/4030274493 conn(0x7f24e00facd0 msgr2=0x7f24e00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.115+0000 7f24e4945700 1 -- 172.21.15.167:0/4030274493 shutdown_connections 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 -- 172.21.15.167:0/4030274493 wait complete. 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 Processor -- start 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 -- start start 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24e01012a0 0x7f24e0195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 0x7f24e019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01966c0 0x7f24e019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e019dfd0 con 0x7f24e01966c0 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e019e140 con 0x7f24e01012a0 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.116+0000 7f24e4945700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e019e300 con 0x7f24e0196240 2024-03-22T19:04:41.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 0x7f24e019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 0x7f24e019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48778/0 (socket says 172.21.15.167:48778) 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 -- 172.21.15.167:0/2413911518 learned_addr learned my addr 172.21.15.167:0/2413911518 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24de59c700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24e01012a0 0x7f24e0195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ded9d700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01966c0 0x7f24e019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 -- 172.21.15.167:0/2413911518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24e01012a0 msgr2=0x7f24e0195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24e01012a0 0x7f24e0195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 -- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01966c0 msgr2=0x7f24e019d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01966c0 0x7f24e019d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 -- 172.21.15.167:0/2413911518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00095f0 con 0x7f24e0196240 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ded9d700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01966c0 0x7f24e019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24ddd9b700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 0x7f24e019b250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f24d400b780 tx=0x7f24d400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4004230 con 0x7f24e0196240 2024-03-22T19:04:41.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e019e5e0 con 0x7f24e0196240 2024-03-22T19:04:41.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d4004390 con 0x7f24e0196240 2024-03-22T19:04:41.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4011ad0 con 0x7f24e0196240 2024-03-22T19:04:41.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.117+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e019ebb0 con 0x7f24e0196240 2024-03-22T19:04:41.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.118+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e018f4e0 con 0x7f24e0196240 2024-03-22T19:04:41.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.118+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f24d4011c30 con 0x7f24e0196240 2024-03-22T19:04:41.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.119+0000 7f24cf7fe700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24c80389f0 0x7f24c803aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:41.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.119+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f24d404a9e0 con 0x7f24e0196240 2024-03-22T19:04:41.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.119+0000 7f24de59c700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24c80389f0 0x7f24c803aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:41.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.120+0000 7f24de59c700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24c80389f0 0x7f24c803aeb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f24e00fd690 tx=0x7f24d0005200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:41.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.124+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f24d401a400 con 0x7f24e0196240 2024-03-22T19:04:41.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: Reconfiguring mon.b (monmap changed)... 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: Reconfiguring daemon mon.b on smithi131 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: Reconfiguring mon.b (monmap changed)... 2024-03-22T19:04:41.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:41.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:41.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:41.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: Reconfiguring daemon mon.b on smithi131 2024-03-22T19:04:41.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:41.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.350+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi131=y;smithi167=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f24e00611f0 con 0x7f24c80389f0 2024-03-22T19:04:41.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.357+0000 7f24cf7fe700 1 -- 172.21.15.167:0/2413911518 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f24e00611f0 con 0x7f24c80389f0 2024-03-22T19:04:41.358 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24c80389f0 msgr2=0x7f24c803aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24c80389f0 0x7f24c803aeb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f24e00fd690 tx=0x7f24d0005200 comp rx=0 tx=0).stop 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 msgr2=0x7f24e019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 0x7f24e019b250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f24d400b780 tx=0x7f24d400ba90 comp rx=0 tx=0).stop 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 shutdown_connections 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24c80389f0 0x7f24c803aeb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e01966c0 0x7f24e019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24e01012a0 0x7f24e0195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 --2- 172.21.15.167:0/2413911518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e0196240 0x7f24e019b250 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:41.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 >> 172.21.15.167:0/2413911518 conn(0x7f24e00facd0 msgr2=0x7f24e00fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:41.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 shutdown_connections 2024-03-22T19:04:41.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:04:41.359+0000 7f24e4945700 1 -- 172.21.15.167:0/2413911518 wait complete. 2024-03-22T19:04:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: Reconfiguring mon.b (monmap changed)... 2024-03-22T19:04:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: Reconfiguring daemon mon.b on smithi131 2024-03-22T19:04:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:41.873 DEBUG:teuthology.orchestra.run.smithi131:mgr.y> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.y.service 2024-03-22T19:04:41.875 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.z.service 2024-03-22T19:04:41.877 INFO:tasks.cephadm:Deploying OSDs... 2024-03-22T19:04:41.878 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:04:41.878 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T19:04:41.906 INFO:journalctl@ceph.mgr.z.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:04:41.910 INFO:journalctl@ceph.mgr.y.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:04:41.924 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T19:04:41.924 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-03-22T19:04:42.011 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T19:04:42.011 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:42.011 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-22T19:04:42.011 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:42.011 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:42.011 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 19:04:01.517242945 +0000 2024-03-22T19:04:42.012 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 18:57:34.207520040 +0000 2024-03-22T19:04:42.012 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 18:57:34.207520040 +0000 2024-03-22T19:04:42.012 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 18:57:34.181520606 +0000 2024-03-22T19:04:42.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T19:04:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T19:04:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T19:04:42.116 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000324164 s, 1.6 MB/s 2024-03-22T19:04:42.118 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T19:04:42.162 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-03-22T19:04:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: Reconfiguring mon.c (monmap changed)... 2024-03-22T19:04:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: Reconfiguring daemon mon.c on smithi167 2024-03-22T19:04:42.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi131=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:42.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: Saving service mgr spec with placement smithi086=x;smithi131=y;smithi167=z;count:3 2024-03-22T19:04:42.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T19:04:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 19:04:01.593241296 +0000 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 18:57:34.674509870 +0000 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 18:57:34.674509870 +0000 2024-03-22T19:04:42.246 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 18:57:34.649510414 +0000 2024-03-22T19:04:42.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T19:04:42.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: Reconfiguring mon.c (monmap changed)... 2024-03-22T19:04:42.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:42.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:42.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:42.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: Reconfiguring daemon mon.c on smithi167 2024-03-22T19:04:42.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi131=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:42.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: Saving service mgr spec with placement smithi086=x;smithi131=y;smithi167=z;count:3 2024-03-22T19:04:42.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T19:04:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T19:04:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000464245 s, 1.1 MB/s 2024-03-22T19:04:42.322 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T19:04:42.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: Reconfiguring mon.c (monmap changed)... 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: Reconfiguring daemon mon.c on smithi167 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi131=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: Saving service mgr spec with placement smithi086=x;smithi131=y;smithi167=z;count:3 2024-03-22T19:04:42.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:42.365 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-03-22T19:04:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 19:04:01.671239602 +0000 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 18:57:35.145499613 +0000 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 18:57:35.145499613 +0000 2024-03-22T19:04:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 18:57:35.122500114 +0000 2024-03-22T19:04:42.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T19:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T19:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T19:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000299382 s, 1.7 MB/s 2024-03-22T19:04:42.552 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T19:04:42.594 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-03-22T19:04:42.677 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T19:04:42.677 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:42.677 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-22T19:04:42.677 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:42.678 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:42.678 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 19:04:01.751237866 +0000 2024-03-22T19:04:42.678 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 18:57:35.623489205 +0000 2024-03-22T19:04:42.678 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 18:57:35.623489205 +0000 2024-03-22T19:04:42.678 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 18:57:35.600489706 +0000 2024-03-22T19:04:42.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T19:04:42.778 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T19:04:42.778 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T19:04:42.778 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000319755 s, 1.6 MB/s 2024-03-22T19:04:42.781 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T19:04:42.824 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-03-22T19:04:42.825 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T19:04:42.846 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T19:04:42.846 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 209514 Links: 1 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-03-22 19:04:30.661835034 +0000 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-03-22 18:56:27.418779941 +0000 2024-03-22T19:04:42.906 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-03-22 18:56:27.418779941 +0000 2024-03-22T19:04:42.907 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-03-22T19:04:42.907 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T19:04:42.975 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-03-22T19:04:42.975 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-03-22T19:04:42.975 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000281166 s, 1.8 MB/s 2024-03-22T19:04:42.976 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T19:04:43.035 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-03-22T19:04:43.094 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 208461 Links: 1 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-03-22 19:04:30.661835034 +0000 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-03-22 18:56:27.759778477 +0000 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-03-22 18:56:27.759778477 +0000 2024-03-22T19:04:43.095 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-03-22T19:04:43.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T19:04:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-03-22T19:04:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-03-22T19:04:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000293222 s, 1.7 MB/s 2024-03-22T19:04:43.164 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T19:04:43.223 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 210785 Links: 1 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-03-22 19:04:30.661835034 +0000 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-03-22 18:56:28.103776999 +0000 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-03-22 18:56:28.103776999 +0000 2024-03-22T19:04:43.282 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-03-22T19:04:43.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T19:04:43.347 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:43.347 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:43.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:43 smithi131 ceph-mon[78559]: Deploying daemon mgr.z on smithi167 2024-03-22T19:04:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-03-22T19:04:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-03-22T19:04:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000172217 s, 3.0 MB/s 2024-03-22T19:04:43.351 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T19:04:43.410 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 214152 Links: 1 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-03-22 19:04:30.661835034 +0000 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-03-22 18:56:28.447775522 +0000 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-03-22 18:56:28.447775522 +0000 2024-03-22T19:04:43.468 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-03-22T19:04:43.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T19:04:43.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:43 smithi086 ceph-mon[38032]: Deploying daemon mgr.z on smithi167 2024-03-22T19:04:43.535 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-03-22T19:04:43.535 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-03-22T19:04:43.535 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000236097 s, 2.2 MB/s 2024-03-22T19:04:43.536 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T19:04:43.578 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:43 smithi167 systemd[1]: Starting Ceph mgr.z for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:04:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:43.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mon[47728]: Deploying daemon mgr.z on smithi167 2024-03-22T19:04:43.595 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-03-22T19:04:43.595 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T19:04:43.613 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T19:04:43.613 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 178653 Links: 1 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-03-22 19:04:43.301332169 +0000 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-03-22 18:56:22.027355219 +0000 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-03-22 18:56:22.027355219 +0000 2024-03-22T19:04:43.672 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-03-22T19:04:43.673 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T19:04:43.742 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-03-22T19:04:43.742 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-03-22T19:04:43.742 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000248212 s, 2.1 MB/s 2024-03-22T19:04:43.743 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T19:04:43.804 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-03-22T19:04:43.861 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T19:04:43.861 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 178706 Links: 1 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-03-22 19:04:43.301332169 +0000 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-03-22 18:56:22.384348113 +0000 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-03-22 18:56:22.384348113 +0000 2024-03-22T19:04:43.862 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-03-22T19:04:43.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T19:04:43.928 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-03-22T19:04:43.928 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-03-22T19:04:43.928 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000222563 s, 2.3 MB/s 2024-03-22T19:04:43.929 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T19:04:43.987 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 178074 Links: 1 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-03-22 19:04:43.301332169 +0000 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-03-22 18:56:22.724341346 +0000 2024-03-22T19:04:44.045 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-03-22 18:56:22.724341346 +0000 2024-03-22T19:04:44.046 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-03-22T19:04:44.046 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T19:04:44.078 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:43 smithi167 ceph-mgr[49658]: -- 172.21.15.167:0/54489979 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c86c7e1a0 con 0x559c86c55000 2024-03-22T19:04:44.078 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:43 smithi167 bash[49561]: a4e12fb0f44322216c600a6e6a344f24f3eb19dc6034775ec4588de4d10bfcf6 2024-03-22T19:04:44.078 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:43 smithi167 systemd[1]: Started Ceph mgr.z for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:04:44.102 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-03-22T19:04:44.103 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-03-22T19:04:44.103 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000192036 s, 2.7 MB/s 2024-03-22T19:04:44.104 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T19:04:44.162 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-03-22T19:04:44.219 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 178166 Links: 1 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-03-22 19:04:43.301332169 +0000 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-03-22 18:56:23.097333922 +0000 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-03-22 18:56:23.097333922 +0000 2024-03-22T19:04:44.220 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-03-22T19:04:44.220 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T19:04:44.279 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:44.180+0000 7fad84abe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T19:04:44.285 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-03-22T19:04:44.286 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-03-22T19:04:44.286 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000229536 s, 2.2 MB/s 2024-03-22T19:04:44.287 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T19:04:44.345 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-03-22T19:04:44.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T19:04:44.370 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:44.278+0000 7fad84abe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T19:04:44.370 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:44.369+0000 7fad84abe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T19:04:44.578 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:44.453+0000 7fad84abe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T19:04:44.697 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:45.214 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: Deploying daemon mgr.y on smithi131 2024-03-22T19:04:45.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:44 smithi086 ceph-mon[38032]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:45.221 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:45 smithi131 systemd[1]: Starting Ceph mgr.y for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: Deploying daemon mgr.y on smithi131 2024-03-22T19:04:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:44 smithi131 ceph-mon[78559]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.085+0000 7fad84abe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: Deploying daemon mgr.y on smithi131 2024-03-22T19:04:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:44 smithi167 ceph-mon[47728]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:45.686 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.334+0000 7fad84abe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T19:04:45.687 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.419+0000 7fad84abe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T19:04:45.687 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.526+0000 7fad84abe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T19:04:45.687 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.685+0000 7fad84abe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T19:04:45.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:45 smithi131 ceph-mgr[80037]: -- 172.21.15.131:0/479674040 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564cd16221a0 con 0x564cd15f8800 2024-03-22T19:04:45.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:45 smithi131 bash[79999]: 33c0402d0c6cb8c39aeb1e0e27f1f518f1b834159b755780a99bef69d48edde7 2024-03-22T19:04:45.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:45 smithi131 systemd[1]: Started Ceph mgr.y for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:04:45.995 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:45 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:45.895+0000 7fb91d785200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T19:04:46.077 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.812+0000 7fad84abe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T19:04:46.078 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:45.906+0000 7fad84abe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T19:04:46.191 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:45 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:45.993+0000 7fb91d785200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T19:04:46.191 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:46.087+0000 7fb91d785200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T19:04:46.191 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:46.189+0000 7fb91d785200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T19:04:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T19:04:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T19:04:46.484 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-22T19:04:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-22T19:04:46.484 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0377601 s, 278 MB/s 2024-03-22T19:04:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-22T19:04:46.578 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:46.324+0000 7fad84abe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T19:04:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:46.864 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.864 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.864 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.864 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.864 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:46.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:46.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:46.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-03-22T19:04:47.032 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:46.741+0000 7fad84abe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T19:04:47.033 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:46.939+0000 7fad84abe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-22T19:04:47.033 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:47.032+0000 7fad84abe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T19:04:47.196 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:46 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:46.862+0000 7fb91d785200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:04:47.196 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:47.111+0000 7fb91d785200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T19:04:47.196 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:47.195+0000 7fb91d785200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T19:04:47.460 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:47.301+0000 7fb91d785200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T19:04:47.460 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:47.459+0000 7fb91d785200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T19:04:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:04:47.721 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:47.586+0000 7fb91d785200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T19:04:47.721 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:47.672+0000 7fb91d785200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T19:04:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:47.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:47.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:47.789+0000 7fad84abe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:47.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:47.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T19:04:47.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T19:04:47.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:48.328 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:47 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:47.886+0000 7fad84abe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T19:04:48.328 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:48.032+0000 7fad84abe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T19:04:48.433 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:48.040+0000 7fb91d785200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: Reconfiguring mgr.x (unknown last config time)... 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: Reconfiguring daemon mgr.x on smithi086 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:48.716 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.717 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:48.431+0000 7fb91d785200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T19:04:48.717 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:48.623+0000 7fb91d785200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-22T19:04:48.717 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:48 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:48.714+0000 7fb91d785200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T19:04:48.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: Reconfiguring mgr.x (unknown last config time)... 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: Reconfiguring daemon mgr.x on smithi086 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:48.664+0000 7fad84abe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: Reconfiguring mgr.x (unknown last config time)... 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: Reconfiguring daemon mgr.x on smithi086 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:04:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:48.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:04:48.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 -- 172.21.15.86:0/3716439394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 msgr2=0x7f67e8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 --2- 172.21.15.86:0/3716439394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e8073c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009990 tx=0x7f67d4009ca0 comp rx=0 tx=0).stop 2024-03-22T19:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 -- 172.21.15.86:0/3716439394 shutdown_connections 2024-03-22T19:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 --2- 172.21.15.86:0/3716439394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67e810d3c0 0x7f67e810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 --2- 172.21.15.86:0/3716439394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67e8074de0 0x7f67e8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 --2- 172.21.15.86:0/3716439394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e8073c90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 -- 172.21.15.86:0/3716439394 >> 172.21.15.86:0/3716439394 conn(0x7f67e80fc380 msgr2=0x7f67e80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 -- 172.21.15.86:0/3716439394 shutdown_connections 2024-03-22T19:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.004+0000 7f67eea00700 1 -- 172.21.15.86:0/3716439394 wait complete. 2024-03-22T19:04:49.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.005+0000 7f67eea00700 1 Processor -- start 2024-03-22T19:04:49.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 -- start start 2024-03-22T19:04:49.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:49.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67e8074de0 0x7f67e819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67e810d3c0 0x7f67e81a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81a4f00 con 0x7f67e8074de0 2024-03-22T19:04:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81a5040 con 0x7f67e8073810 2024-03-22T19:04:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eea00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81a51d0 con 0x7f67e810d3c0 2024-03-22T19:04:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67e8074de0 0x7f67e819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67eca00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67e810d3c0 0x7f67e81a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e819d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43362/0 (socket says 172.21.15.86:43362) 2024-03-22T19:04:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 -- 172.21.15.86:0/3028006690 learned_addr learned my addr 172.21.15.86:0/3028006690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:04:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 -- 172.21.15.86:0/3028006690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67e810d3c0 msgr2=0x7f67e81a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67e810d3c0 0x7f67e81a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 -- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67e8074de0 msgr2=0x7f67e819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:04:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67e8074de0 0x7f67e819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:04:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.006+0000 7f67e7e00700 1 -- 172.21.15.86:0/3028006690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d40095f0 con 0x7f67e8073810 2024-03-22T19:04:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.007+0000 7f67e7e00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e819d3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67d800e970 tx=0x7f67d800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.007+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8009790 con 0x7f67e8073810 2024-03-22T19:04:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.007+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d800c7e0 con 0x7f67e8073810 2024-03-22T19:04:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.007+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80054e0 con 0x7f67e8073810 2024-03-22T19:04:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.007+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e81a54b0 con 0x7f67e8073810 2024-03-22T19:04:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.007+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e81a5960 con 0x7f67e8073810 2024-03-22T19:04:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.008+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e8103620 con 0x7f67e8073810 2024-03-22T19:04:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.008+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45430+0+0 (secure 0 0 0) 0x7f67d800c950 con 0x7f67e8073810 2024-03-22T19:04:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.009+0000 7f67e4c00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67cc0387a0 0x7f67cc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:04:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.009+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f67d8014070 con 0x7f67e8073810 2024-03-22T19:04:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.009+0000 7f67e7400700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67cc0387a0 0x7f67cc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:04:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.010+0000 7f67e7400700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67cc0387a0 0x7f67cc03ac60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000c00 tx=0x7f67d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:04:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.014+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f67d801f520 con 0x7f67e8073810 2024-03-22T19:04:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:49.224+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f67e80611f0 con 0x7f67cc0387a0 2024-03-22T19:04:49.328 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:48 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:48.843+0000 7fad84abe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T19:04:49.328 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:49 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:49.034+0000 7fad84abe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T19:04:49.827 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:49 smithi086 ceph-mon[38032]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:49.827 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:04:49.827 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:04:49.827 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:49 smithi167 ceph-mon[47728]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:04:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:04:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-mon[78559]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:04:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:04:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:49.972 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:49.471+0000 7fb91d785200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T19:04:49.972 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:49.565+0000 7fb91d785200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T19:04:49.972 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:49 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:49.712+0000 7fb91d785200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T19:04:50.239 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:49 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:49.956+0000 7fad84abe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T19:04:50.239 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:50.238+0000 7fad84abe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:04:50.517 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:50.326+0000 7fad84abe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T19:04:50.707 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:50.707 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:50.707 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: Standby manager daemon z started 2024-03-22T19:04:50.707 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-22T19:04:50.707 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T19:04:50.708 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-22T19:04:50.708 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T19:04:50.708 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:50.334+0000 7fb91d785200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-22T19:04:50.708 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:50.514+0000 7fb91d785200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T19:04:50.708 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:50 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:50.706+0000 7fb91d785200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T19:04:50.828 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z[49576]: 2024-03-22T19:04:50.516+0000 7fad84abe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T19:04:50.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: Standby manager daemon z started 2024-03-22T19:04:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-22T19:04:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T19:04:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-22T19:04:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:50 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T19:04:51.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:04:51.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:04:51.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: Standby manager daemon z started 2024-03-22T19:04:51.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-22T19:04:51.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T19:04:51.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-22T19:04:51.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:50 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.167:0/678232048' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T19:04:51.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:51.456+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 14) v1 ==== 90381+0+0 (secure 0 0 0) 0x7f67d8021440 con 0x7f67e8073810 2024-03-22T19:04:51.914 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:51.631+0000 7fb91d785200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T19:04:51.914 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:51.914 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-22T19:04:51.914 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-22T19:04:51.914 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/309381887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edf49b32-3333-4f40-bf4d-fb6ac26e3890"}]: dispatch 2024-03-22T19:04:51.915 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/309381887' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edf49b32-3333-4f40-bf4d-fb6ac26e3890"}]': finished 2024-03-22T19:04:51.915 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: osdmap e5: 1 total, 0 up, 1 in 2024-03-22T19:04:51.915 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:04:52.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:52.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-22T19:04:52.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-22T19:04:52.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/309381887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edf49b32-3333-4f40-bf4d-fb6ac26e3890"}]: dispatch 2024-03-22T19:04:52.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/309381887' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edf49b32-3333-4f40-bf4d-fb6ac26e3890"}]': finished 2024-03-22T19:04:52.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: osdmap e5: 1 total, 0 up, 1 in 2024-03-22T19:04:52.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/309381887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edf49b32-3333-4f40-bf4d-fb6ac26e3890"}]: dispatch 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/309381887' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edf49b32-3333-4f40-bf4d-fb6ac26e3890"}]': finished 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: osdmap e5: 1 total, 0 up, 1 in 2024-03-22T19:04:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:04:52.189 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:51 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:51.912+0000 7fb91d785200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T19:04:52.189 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:51.999+0000 7fb91d785200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T19:04:52.189 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y[80014]: 2024-03-22T19:04:52.188+0000 7fb91d785200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T19:04:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:04:52.591+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mon.2 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 15) v1 ==== 135332+0+0 (secure 0 0 0) 0x7f67d8004a30 con 0x7f67e8073810 2024-03-22T19:04:52.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-mon[78559]: Standby manager daemon y started 2024-03-22T19:04:52.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-22T19:04:52.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T19:04:52.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-22T19:04:52.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-mon[78559]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T19:04:52.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:52 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/974444169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:04:53.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:52 smithi086 ceph-mon[38032]: Standby manager daemon y started 2024-03-22T19:04:53.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:52 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-22T19:04:53.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:52 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T19:04:53.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:52 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-22T19:04:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:52 smithi086 ceph-mon[38032]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T19:04:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:52 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/974444169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:04:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:52 smithi167 ceph-mon[47728]: Standby manager daemon y started 2024-03-22T19:04:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:52 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-22T19:04:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:52 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T19:04:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:52 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-22T19:04:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:52 smithi167 ceph-mon[47728]: from='mgr.? 172.21.15.131:0/1690473229' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T19:04:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:52 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/974444169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:04:53.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:53 smithi131 ceph-mon[78559]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:53.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:53 smithi131 ceph-mon[78559]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-03-22T19:04:53.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-22T19:04:54.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:53 smithi086 ceph-mon[38032]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:54.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:53 smithi086 ceph-mon[38032]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-03-22T19:04:54.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-22T19:04:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:53 smithi167 ceph-mon[47728]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:53 smithi167 ceph-mon[47728]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-03-22T19:04:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-22T19:04:55.864 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:55 smithi086 ceph-mon[38032]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:56.008 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:55 smithi131 ceph-mon[78559]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:56.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:55 smithi167 ceph-mon[47728]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:57.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:57 smithi086 ceph-mon[38032]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:57.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:57 smithi131 ceph-mon[78559]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:58.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:57 smithi167 ceph-mon[47728]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:58.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:58 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T19:04:58.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:58 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:59.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:58 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T19:04:59.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:58 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:59.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:58 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T19:04:59.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:58 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:04:59.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:59 smithi086 ceph-mon[38032]: Deploying daemon osd.0 on smithi086 2024-03-22T19:04:59.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:04:59 smithi086 ceph-mon[38032]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:04:59.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:59 smithi131 ceph-mon[78559]: Deploying daemon osd.0 on smithi086 2024-03-22T19:04:59.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:04:59 smithi131 ceph-mon[78559]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:00.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:59 smithi167 ceph-mon[47728]: Deploying daemon osd.0 on smithi086 2024-03-22T19:05:00.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:04:59 smithi167 ceph-mon[47728]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:01 smithi131 ceph-mon[78559]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:01.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:01 smithi086 ceph-mon[38032]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:01 smithi167 ceph-mon[47728]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:03 smithi131 ceph-mon[78559]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:03.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:03.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:03 smithi086 ceph-mon[38032]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:03.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:03.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:03.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:03 smithi167 ceph-mon[47728]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:05 smithi131 ceph-mon[78559]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:05.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:05 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:05.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:05 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:05.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:05 smithi086 ceph-mon[38032]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:05.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:05.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:05 smithi167 ceph-mon[47728]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:05 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:05 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:05.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.980+0000 7f67e4c00700 1 -- 172.21.15.86:0/3028006690 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f67e80611f0 con 0x7f67cc0387a0 2024-03-22T19:05:05.981 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-03-22T19:05:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67cc0387a0 msgr2=0x7f67cc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67cc0387a0 0x7f67cc03ac60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000c00 tx=0x7f67d4005fb0 comp rx=0 tx=0).stop 2024-03-22T19:05:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 msgr2=0x7f67e819d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e819d3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67d800e970 tx=0x7f67d800ec80 comp rx=0 tx=0).stop 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 shutdown_connections 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67cc0387a0 0x7f67cc03ac60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67e810d3c0 0x7f67e81a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67e8073810 0x7f67e819d3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 --2- 172.21.15.86:0/3028006690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67e8074de0 0x7f67e819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.982+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 >> 172.21.15.86:0/3028006690 conn(0x7f67e80fc380 msgr2=0x7f67e8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.983+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 shutdown_connections 2024-03-22T19:05:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:05.983+0000 7f67eea00700 1 -- 172.21.15.86:0/3028006690 wait complete. 2024-03-22T19:05:06.396 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.0.service 2024-03-22T19:05:06.400 INFO:tasks.cephadm:Deploying osd.1 on smithi131 with /dev/vg_nvme/lv_4... 2024-03-22T19:05:06.400 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T19:05:06.472 INFO:journalctl@ceph.osd.0.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:05:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:05:06.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:06.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:06.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:07.008 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:06 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0[56811]: 2024-03-22T19:05:06.815+0000 7f76b66bd640 -1 osd.0 0 log_to_monitors true 2024-03-22T19:05:07.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:07 smithi086 ceph-mon[38032]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:07.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:07 smithi086 ceph-mon[38032]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T19:05:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:07 smithi167 ceph-mon[47728]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:07 smithi167 ceph-mon[47728]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T19:05:07.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:07 smithi131 ceph-mon[78559]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:07.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:07 smithi131 ceph-mon[78559]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T19:05:08.018 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T19:05:08.019 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T19:05:08.019 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-03-22T19:05:08.019 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-03-22T19:05:08.019 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112619 s, 931 MB/s 2024-03-22T19:05:08.019 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-03-22T19:05:08.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-03-22T19:05:08.636 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:05:08.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T19:05:08.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T19:05:08.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:08.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:05:08.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:05:08.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T19:05:08.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:08 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T19:05:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T19:05:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:05:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:05:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T19:05:08.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:08.760 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0[56811]: 2024-03-22T19:05:08.486+0000 7f76aac00700 -1 osd.0 0 waiting for initial osdmap 2024-03-22T19:05:08.760 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:08 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0[56811]: 2024-03-22T19:05:08.490+0000 7f76a3200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: from='osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T19:05:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:08 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:09.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 -- 172.21.15.131:0/1200239204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102ad0 msgr2=0x7fd434109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:09.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 --2- 172.21.15.131:0/1200239204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102ad0 0x7fd434109380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b0a0 tx=0x7fd42c00b3b0 comp rx=0 tx=0).stop 2024-03-22T19:05:09.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 -- 172.21.15.131:0/1200239204 shutdown_connections 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 --2- 172.21.15.131:0/1200239204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102ad0 0x7fd434109380 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 --2- 172.21.15.131:0/1200239204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4340ff0b0 0x7fd4340ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 --2- 172.21.15.131:0/1200239204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd434102130 0x7fd434102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 -- 172.21.15.131:0/1200239204 >> 172.21.15.131:0/1200239204 conn(0x7fd4340fa9f0 msgr2=0x7fd4340fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 -- 172.21.15.131:0/1200239204 shutdown_connections 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.577+0000 7fd43bf77700 1 -- 172.21.15.131:0/1200239204 wait complete. 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 Processor -- start 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 -- start start 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4340ff0b0 0x7fd434198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 0x7fd4341994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd434102ad0 0x7fd4341a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4341a0af0 con 0x7fd4340ff0b0 2024-03-22T19:05:09.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4341a0c30 con 0x7fd434102130 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43bf77700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4341a0dc0 con 0x7fd434102ad0 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 0x7fd4341994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 0x7fd4341994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60828/0 (socket says 172.21.15.131:60828) 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 -- 172.21.15.131:0/4162509592 learned_addr learned my addr 172.21.15.131:0/4162509592 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd43a514700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd434102ad0 0x7fd4341a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 -- 172.21.15.131:0/4162509592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd434102ad0 msgr2=0x7fd4341a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd434102ad0 0x7fd4341a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 -- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4340ff0b0 msgr2=0x7fd434198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4340ff0b0 0x7fd434198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 -- 172.21.15.131:0/4162509592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c009d00 con 0x7fd434102130 2024-03-22T19:05:09.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.578+0000 7fd439512700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 0x7fd4341994f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd43000d950 tx=0x7fd43000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:05:09.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.579+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430009940 con 0x7fd434102130 2024-03-22T19:05:09.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.579+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd430010460 con 0x7fd434102130 2024-03-22T19:05:09.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.579+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4341a10a0 con 0x7fd434102130 2024-03-22T19:05:09.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.579+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43000f600 con 0x7fd434102130 2024-03-22T19:05:09.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.579+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4341a15f0 con 0x7fd434102130 2024-03-22T19:05:09.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.580+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43404ea80 con 0x7fd434102130 2024-03-22T19:05:09.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.580+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) v1 ==== 135332+0+0 (secure 0 0 0) 0x7fd4300105d0 con 0x7fd434102130 2024-03-22T19:05:09.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.581+0000 7fd42affd700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd4200a0a80 0x7fd4200a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:09.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.581+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd4300ca380 con 0x7fd434102130 2024-03-22T19:05:09.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.581+0000 7fd439d13700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd4200a0a80 0x7fd4200a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:09.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.582+0000 7fd439d13700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd4200a0a80 0x7fd4200a2f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd434068460 tx=0x7fd424009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:05:09.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.585+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd4300957c0 con 0x7fd434102130 2024-03-22T19:05:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:09.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:09.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:09.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:09.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:09.784+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd434101640 con 0x7fd4200a0a80 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:10.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi086 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi086 to 17270M 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] boot 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: osdmap e8: 1 total, 1 up, 1 in 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi086 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi086 to 17270M 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] boot 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: osdmap e8: 1 total, 1 up, 1 in 2024-03-22T19:05:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:10.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:10.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:10.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi086 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi086 to 17270M 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: osd.0 [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] boot 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: osdmap e8: 1 total, 1 up, 1 in 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:10.871 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:11.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:11 smithi086 ceph-mon[38032]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:11.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:11 smithi086 ceph-mon[38032]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:11.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:11 smithi086 ceph-mon[38032]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T19:05:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:11 smithi167 ceph-mon[47728]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:11 smithi167 ceph-mon[47728]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:11 smithi167 ceph-mon[47728]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T19:05:11.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:11 smithi131 ceph-mon[78559]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:11.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:11 smithi131 ceph-mon[78559]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T19:05:11.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:11 smithi131 ceph-mon[78559]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T19:05:12.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/31028883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]: dispatch 2024-03-22T19:05:12.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:12 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]: dispatch 2024-03-22T19:05:12.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:12 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]': finished 2024-03-22T19:05:12.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:12 smithi086 ceph-mon[38032]: osdmap e10: 2 total, 1 up, 2 in 2024-03-22T19:05:12.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:12 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:12.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/4057509933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/31028883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]: dispatch 2024-03-22T19:05:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:12 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]: dispatch 2024-03-22T19:05:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:12 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]': finished 2024-03-22T19:05:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:12 smithi167 ceph-mon[47728]: osdmap e10: 2 total, 1 up, 2 in 2024-03-22T19:05:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:12 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/4057509933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:12.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/31028883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]: dispatch 2024-03-22T19:05:12.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:12 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]: dispatch 2024-03-22T19:05:12.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:12 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dc78998-5904-4af2-97fd-880603deeb09"}]': finished 2024-03-22T19:05:12.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:12 smithi131 ceph-mon[78559]: osdmap e10: 2 total, 1 up, 2 in 2024-03-22T19:05:12.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:12 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:12.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/4057509933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:13.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:13 smithi086 ceph-mon[38032]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:13 smithi167 ceph-mon[47728]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:13.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:13 smithi131 ceph-mon[78559]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:15.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:15 smithi131 ceph-mon[78559]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:15 smithi167 ceph-mon[47728]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:16.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:15 smithi086 ceph-mon[38032]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:17.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:17 smithi131 ceph-mon[78559]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:17.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T19:05:17.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:17 smithi167 ceph-mon[47728]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T19:05:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:18.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:17 smithi086 ceph-mon[38032]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:18.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T19:05:18.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:18 smithi167 ceph-mon[47728]: Deploying daemon osd.1 on smithi131 2024-03-22T19:05:18.914 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:18 smithi131 ceph-mon[78559]: Deploying daemon osd.1 on smithi131 2024-03-22T19:05:19.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:18 smithi086 ceph-mon[38032]: Deploying daemon osd.1 on smithi131 2024-03-22T19:05:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:19 smithi167 ceph-mon[47728]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:19.889 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:19 smithi131 ceph-mon[78559]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:19 smithi086 ceph-mon[38032]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:21 smithi167 ceph-mon[47728]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:21.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:21 smithi131 ceph-mon[78559]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:22.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:21 smithi086 ceph-mon[38032]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:22.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:23.164 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.164 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.164 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:23 smithi131 ceph-mon[78559]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:23.164 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:23.164 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:23.164 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.298+0000 7fd42affd700 1 -- 172.21.15.131:0/4162509592 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd434101640 con 0x7fd4200a0a80 2024-03-22T19:05:23.300 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 1 on host 'smithi131' 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd4200a0a80 msgr2=0x7fd4200a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd4200a0a80 0x7fd4200a2f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd434068460 tx=0x7fd424009470 comp rx=0 tx=0).stop 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 msgr2=0x7fd4341994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 0x7fd4341994f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd43000d950 tx=0x7fd43000dc60 comp rx=0 tx=0).stop 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 shutdown_connections 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd4200a0a80 0x7fd4200a2f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd434102ad0 0x7fd4341a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4340ff0b0 0x7fd434198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 --2- 172.21.15.131:0/4162509592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd434102130 0x7fd4341994f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 >> 172.21.15.131:0/4162509592 conn(0x7fd4340fa9f0 msgr2=0x7fd4340fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:05:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 shutdown_connections 2024-03-22T19:05:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:05:23.300+0000 7fd43bf77700 1 -- 172.21.15.131:0/4162509592 wait complete. 2024-03-22T19:05:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:23 smithi086 ceph-mon[38032]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:23 smithi167 ceph-mon[47728]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:23.691 DEBUG:teuthology.orchestra.run.smithi131:osd.1> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.1.service 2024-03-22T19:05:23.693 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-03-22T19:05:23.694 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T19:05:23.722 INFO:journalctl@ceph.osd.1.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:05:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:05:24.469 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:24.470 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:24.470 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:24.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:24.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:24.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:25.396 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T19:05:25.396 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T19:05:25.396 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-03-22T19:05:25.396 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-03-22T19:05:25.396 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112802 s, 930 MB/s 2024-03-22T19:05:25.396 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-03-22T19:05:25.471 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:25 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1[83530]: 2024-03-22T19:05:25.042+0000 7f379dad0640 -1 osd.1 0 log_to_monitors true 2024-03-22T19:05:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:25 smithi131 ceph-mon[78559]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:25.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:25 smithi131 ceph-mon[78559]: from='osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T19:05:25.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:25 smithi131 ceph-mon[78559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T19:05:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:25 smithi167 ceph-mon[47728]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:25 smithi167 ceph-mon[47728]: from='osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T19:05:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:25 smithi167 ceph-mon[47728]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T19:05:25.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:25 smithi086 ceph-mon[38032]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:25.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:25 smithi086 ceph-mon[38032]: from='osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T19:05:25.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:25 smithi086 ceph-mon[38032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T19:05:25.787 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-03-22T19:05:26.004 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:05:26.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T19:05:26.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T19:05:26.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:26.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:26.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.721 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1[83530]: 2024-03-22T19:05:26.280+0000 7f3793134700 -1 osd.1 0 waiting for initial osdmap 2024-03-22T19:05:26.722 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1[83530]: 2024-03-22T19:05:26.283+0000 7f378af21700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:26.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:26.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:26.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T19:05:26.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:26.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 -- 172.21.15.167:0/474814870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 msgr2=0x7f74e41065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/474814870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e41065e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009990 tx=0x7f74d4009ca0 comp rx=0 tx=0).stop 2024-03-22T19:05:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 -- 172.21.15.167:0/474814870 shutdown_connections 2024-03-22T19:05:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/474814870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74e4106b20 0x7f74e4108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/474814870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e41018c0 0x7f74e4103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/474814870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e41065e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 -- 172.21.15.167:0/474814870 >> 172.21.15.167:0/474814870 conn(0x7f74e40fb270 msgr2=0x7f74e40fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 -- 172.21.15.167:0/474814870 shutdown_connections 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.002+0000 7f74ea8e5700 1 -- 172.21.15.167:0/474814870 wait complete. 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 Processor -- start 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 -- start start 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74e41018c0 0x7f74e4196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e4197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e4106b20 0x7f74e419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e419e830 con 0x7f74e41018c0 2024-03-22T19:05:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e419e970 con 0x7f74e4106b20 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74ea8e5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e419eb30 con 0x7f74e41041f0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e4197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e4197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33732/0 (socket says 172.21.15.167:33732) 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 -- 172.21.15.167:0/586884950 learned_addr learned my addr 172.21.15.167:0/586884950 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e8e82700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e4106b20 0x7f74e419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e3fff700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74e41018c0 0x7f74e4196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 -- 172.21.15.167:0/586884950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e4106b20 msgr2=0x7f74e419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e4106b20 0x7f74e419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 -- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74e41018c0 msgr2=0x7f74e4196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74e41018c0 0x7f74e4196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.003+0000 7f74e37fe700 1 -- 172.21.15.167:0/586884950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d40095f0 con 0x7f74e41041f0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.004+0000 7f74e37fe700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e4197270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f74d4000c00 tx=0x7f74d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.004+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d401d070 con 0x7f74e41041f0 2024-03-22T19:05:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.004+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e419edb0 con 0x7f74e41041f0 2024-03-22T19:05:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.004+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74d40054f0 con 0x7f74e41041f0 2024-03-22T19:05:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.004+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004cd0 con 0x7f74e41041f0 2024-03-22T19:05:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.004+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e419f170 con 0x7f74e41041f0 2024-03-22T19:05:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.005+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e40fcdf0 con 0x7f74e41041f0 2024-03-22T19:05:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.006+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) v1 ==== 135332+0+0 (secure 0 0 0) 0x7f74d400fc40 con 0x7f74e41041f0 2024-03-22T19:05:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.006+0000 7f74e17fa700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f74cc0a0a00 0x7f74cc0a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.006+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f74d40cb9e0 con 0x7f74e41041f0 2024-03-22T19:05:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.007+0000 7f74e3fff700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f74cc0a0a00 0x7f74cc0a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.007+0000 7f74e3fff700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f74cc0a0a00 0x7f74cc0a2ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74e41981c0 tx=0x7f74d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:05:27.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.010+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74d4096db0 con 0x7f74e41041f0 2024-03-22T19:05:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:27.211+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f74e40611f0 con 0x7f74cc0a0a00 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi131 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi131 to 17113M 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:27.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:27.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:27.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:27 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:27.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi131 2024-03-22T19:05:27.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi131 to 17113M 2024-03-22T19:05:27.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:27 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:27.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi131 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi131 to 17113M 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:27 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:28 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:05:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:28 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:05:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:28 smithi167 ceph-mon[47728]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:28 smithi167 ceph-mon[47728]: osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] boot 2024-03-22T19:05:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:28 smithi167 ceph-mon[47728]: osdmap e13: 2 total, 2 up, 2 in 2024-03-22T19:05:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:28 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:28.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:28 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:05:28.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:28 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:05:28.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:28 smithi131 ceph-mon[78559]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:28.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:28 smithi131 ceph-mon[78559]: osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] boot 2024-03-22T19:05:28.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:28 smithi131 ceph-mon[78559]: osdmap e13: 2 total, 2 up, 2 in 2024-03-22T19:05:28.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:28 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:28.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:28 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:05:28.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:28 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:05:28.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:28 smithi086 ceph-mon[38032]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:28.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:28 smithi086 ceph-mon[38032]: osd.1 [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] boot 2024-03-22T19:05:28.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:28 smithi086 ceph-mon[38032]: osdmap e13: 2 total, 2 up, 2 in 2024-03-22T19:05:28.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:28 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T19:05:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T19:05:29.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/3264223180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]: dispatch 2024-03-22T19:05:29.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]: dispatch 2024-03-22T19:05:29.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]': finished 2024-03-22T19:05:29.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: osdmap e15: 3 total, 2 up, 3 in 2024-03-22T19:05:29.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T19:05:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/3264223180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]: dispatch 2024-03-22T19:05:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]: dispatch 2024-03-22T19:05:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]': finished 2024-03-22T19:05:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: osdmap e15: 3 total, 2 up, 3 in 2024-03-22T19:05:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:29 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:29.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T19:05:29.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T19:05:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/3264223180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]: dispatch 2024-03-22T19:05:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]: dispatch 2024-03-22T19:05:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84f39745-e2e8-4b26-8041-28d4e8914192"}]': finished 2024-03-22T19:05:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: osdmap e15: 3 total, 2 up, 3 in 2024-03-22T19:05:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:30.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:30 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/885298228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:30.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:30 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/885298228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:30.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:30 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/885298228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:31 smithi131 ceph-mon[78559]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:31.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:31 smithi086 ceph-mon[38032]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:31.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:31 smithi167 ceph-mon[47728]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:33.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:33 smithi086 ceph-mon[38032]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:33.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:33 smithi167 ceph-mon[47728]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:33.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:33 smithi131 ceph-mon[78559]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:35.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:35 smithi167 ceph-mon[47728]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:35.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:35 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T19:05:35.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:35 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:35 smithi086 ceph-mon[38032]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:35 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T19:05:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:35 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:35.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:35 smithi131 ceph-mon[78559]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:35.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:35 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T19:05:35.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:35 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:36 smithi167 ceph-mon[47728]: Deploying daemon osd.2 on smithi167 2024-03-22T19:05:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:36 smithi086 ceph-mon[38032]: Deploying daemon osd.2 on smithi167 2024-03-22T19:05:36.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:36 smithi131 ceph-mon[78559]: Deploying daemon osd.2 on smithi167 2024-03-22T19:05:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:37 smithi167 ceph-mon[47728]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:37 smithi086 ceph-mon[38032]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:37.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:37 smithi131 ceph-mon[78559]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:39 smithi131 ceph-mon[78559]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:39.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:39 smithi167 ceph-mon[47728]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:39.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:39.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:39.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:39 smithi086 ceph-mon[38032]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:40.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:40.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:40.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:40 smithi167 ceph-mon[47728]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:40.920 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-03-22T19:05:40.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.919+0000 7f74e17fa700 1 -- 172.21.15.167:0/586884950 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f74e40611f0 con 0x7f74cc0a0a00 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f74cc0a0a00 msgr2=0x7f74cc0a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f74cc0a0a00 0x7f74cc0a2ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74e41981c0 tx=0x7f74d8009260 comp rx=0 tx=0).stop 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 msgr2=0x7f74e4197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e4197270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f74d4000c00 tx=0x7f74d4004730 comp rx=0 tx=0).stop 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 shutdown_connections 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f74cc0a0a00 0x7f74cc0a2ec0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:40.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e4106b20 0x7f74e419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:40.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74e41018c0 0x7f74e4196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:40.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 --2- 172.21.15.167:0/586884950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74e41041f0 0x7f74e4197270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:40.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 >> 172.21.15.167:0/586884950 conn(0x7f74e40fb270 msgr2=0x7f74e4103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:05:40.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 shutdown_connections 2024-03-22T19:05:40.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:05:40.921+0000 7f74ea8e5700 1 -- 172.21.15.167:0/586884950 wait complete. 2024-03-22T19:05:40.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:40.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:40.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:40.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:40 smithi131 ceph-mon[78559]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:41.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:40 smithi086 ceph-mon[38032]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:41.317 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.2.service 2024-03-22T19:05:41.319 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-03-22T19:05:41.320 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T19:05:41.351 INFO:journalctl@ceph.osd.2.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:05:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:05:41.943 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:41.943 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:41.943 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:42.181 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:42.181 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:42.181 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:05:42.867 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:42 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2[53482]: 2024-03-22T19:05:42.590+0000 7f79fbcd5640 -1 osd.2 0 log_to_monitors true 2024-03-22T19:05:43.181 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:42 smithi167 ceph-mon[47728]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:43.181 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:42 smithi167 ceph-mon[47728]: from='osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T19:05:43.181 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:42 smithi167 ceph-mon[47728]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T19:05:43.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:42 smithi131 ceph-mon[78559]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:43.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:42 smithi131 ceph-mon[78559]: from='osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T19:05:43.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:42 smithi131 ceph-mon[78559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T19:05:43.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:42 smithi086 ceph-mon[38032]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:43.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:42 smithi086 ceph-mon[38032]: from='osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T19:05:43.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:42 smithi086 ceph-mon[38032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T19:05:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T19:05:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T19:05:43.363 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-22T19:05:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-22T19:05:43.364 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0311342 s, 337 MB/s 2024-03-22T19:05:43.364 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-22T19:05:43.751 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi167 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi167 to 17114M 2024-03-22T19:05:44.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:44.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:44.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:05:44.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T19:05:44.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi167 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:44.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi167 to 17114M 2024-03-22T19:05:44.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:44.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:44.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.328 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2[53482]: 2024-03-22T19:05:43.885+0000 7f79f1339700 -1 osd.2 0 waiting for initial osdmap 2024-03-22T19:05:44.328 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2[53482]: 2024-03-22T19:05:43.889+0000 7f79ed12e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:05:44.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T19:05:44.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T19:05:44.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:44.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:05:44.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi167 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi167 to 17114M 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:05:44.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:45.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:44 smithi131 ceph-mon[78559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:05:45.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:44 smithi131 ceph-mon[78559]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T19:05:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:44 smithi131 ceph-mon[78559]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:45.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:44 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 -- 172.21.15.86:0/2797999809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1041f0 msgr2=0x7f785c1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 --2- 172.21.15.86:0/2797999809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1041f0 0x7f785c1065e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7848009990 tx=0x7f7848009ca0 comp rx=0 tx=0).stop 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 -- 172.21.15.86:0/2797999809 shutdown_connections 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 --2- 172.21.15.86:0/2797999809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f785c106b20 0x7f785c108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 --2- 172.21.15.86:0/2797999809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1018c0 0x7f785c103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 --2- 172.21.15.86:0/2797999809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1041f0 0x7f785c1065e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 -- 172.21.15.86:0/2797999809 >> 172.21.15.86:0/2797999809 conn(0x7f785c0fb270 msgr2=0x7f785c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 -- 172.21.15.86:0/2797999809 shutdown_connections 2024-03-22T19:05:45.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.229+0000 7f7862400700 1 -- 172.21.15.86:0/2797999809 wait complete. 2024-03-22T19:05:45.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.230+0000 7f7862400700 1 Processor -- start 2024-03-22T19:05:45.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.230+0000 7f7862400700 1 -- start start 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f7862400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 0x7f785c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f7862400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1041f0 0x7f785c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f7862400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f785c106b20 0x7f785c1a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f7862400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1a0ad0 con 0x7f785c1018c0 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f7862400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1a0c10 con 0x7f785c106b20 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f7862400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1a0da0 con 0x7f785c1041f0 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f785b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 0x7f785c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f785b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 0x7f785c198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f785b400700 1 -- 172.21.15.86:0/1390397954 learned_addr learned my addr 172.21.15.86:0/1390397954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f785aa00700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1041f0 0x7f785c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:45.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.231+0000 7f785be00700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f785c106b20 0x7f785c1a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785b400700 1 -- 172.21.15.86:0/1390397954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1041f0 msgr2=0x7f785c1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785b400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1041f0 0x7f785c1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785b400700 1 -- 172.21.15.86:0/1390397954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f785c106b20 msgr2=0x7f785c1a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785b400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f785c106b20 0x7f785c1a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785b400700 1 -- 172.21.15.86:0/1390397954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78480095f0 con 0x7f785c1018c0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785aa00700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1041f0 0x7f785c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f785b400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 0x7f785c198f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f784c00d980 tx=0x7f784c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0098d0 con 0x7f785c1018c0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f784c010460 con 0x7f785c1018c0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c1a1080 con 0x7f785c1018c0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00f620 con 0x7f785c1018c0 2024-03-22T19:05:45.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.232+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c1a15d0 con 0x7f785c1018c0 2024-03-22T19:05:45.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.233+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c1086c0 con 0x7f785c1018c0 2024-03-22T19:05:45.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.235+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) v1 ==== 135332+0+0 (secure 0 0 0) 0x7f784c010860 con 0x7f785c1018c0 2024-03-22T19:05:45.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.236+0000 7f7853e00700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78400a0a80 0x7f78400a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:05:45.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.236+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f784c0ca790 con 0x7f785c1018c0 2024-03-22T19:05:45.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.237+0000 7f785aa00700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78400a0a80 0x7f78400a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:05:45.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.237+0000 7f785aa00700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78400a0a80 0x7f78400a2f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f784800b5c0 tx=0x7f7848003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:05:45.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.241+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f784c0956f0 con 0x7f785c1018c0 2024-03-22T19:05:45.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:44 smithi086 ceph-mon[38032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:05:45.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:44 smithi086 ceph-mon[38032]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T19:05:45.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:44 smithi086 ceph-mon[38032]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:45.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:44 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:44 smithi167 ceph-mon[47728]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:05:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:44 smithi167 ceph-mon[47728]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T19:05:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:44 smithi167 ceph-mon[47728]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T19:05:45.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:44 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:45.452+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f785c0611f0 con 0x7f78400a0a80 2024-03-22T19:05:46.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] boot 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: osdmap e18: 3 total, 3 up, 3 in 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:46.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:46.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:05:46.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] boot 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: osdmap e18: 3 total, 3 up, 3 in 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:46.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: osd.2 [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] boot 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: osdmap e18: 3 total, 3 up, 3 in 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:05:46.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:05:46.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:47.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:46 smithi131 ceph-mon[78559]: pgmap v66: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:47.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:05:47.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-22T19:05:47.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:46 smithi131 ceph-mon[78559]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T19:05:47.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:05:47.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:46 smithi086 ceph-mon[38032]: pgmap v66: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:47.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:05:47.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-22T19:05:47.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:46 smithi086 ceph-mon[38032]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T19:05:47.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:05:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:46 smithi167 ceph-mon[47728]: pgmap v66: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:05:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-22T19:05:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:46 smithi167 ceph-mon[47728]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T19:05:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:05:48.704 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T19:05:48.704 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:48.705 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:48.705 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63153]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:05:48.881+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f784c095ac0 con 0x7f785c1018c0 2024-03-22T19:05:48.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T19:05:48.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T19:05:48.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/184535387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c78c338-9e45-43c1-abed-bed5550c0582"}]: dispatch 2024-03-22T19:05:48.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/184535387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c78c338-9e45-43c1-abed-bed5550c0582"}]': finished 2024-03-22T19:05:48.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: osdmap e21: 4 total, 3 up, 4 in 2024-03-22T19:05:48.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:05:48.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:48 smithi131 ceph-mon[78559]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:49.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T19:05:49.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T19:05:49.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/184535387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c78c338-9e45-43c1-abed-bed5550c0582"}]: dispatch 2024-03-22T19:05:49.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/184535387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c78c338-9e45-43c1-abed-bed5550c0582"}]': finished 2024-03-22T19:05:49.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: osdmap e21: 4 total, 3 up, 4 in 2024-03-22T19:05:49.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:05:49.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:48 smithi086 ceph-mon[38032]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:49.009 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63225]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T19:05:49.009 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63225]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:49.009 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63225]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:49.009 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:05:48 smithi086 sudo[63225]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:49.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T19:05:49.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T19:05:49.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/184535387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c78c338-9e45-43c1-abed-bed5550c0582"}]: dispatch 2024-03-22T19:05:49.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/184535387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c78c338-9e45-43c1-abed-bed5550c0582"}]': finished 2024-03-22T19:05:49.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: osdmap e21: 4 total, 3 up, 4 in 2024-03-22T19:05:49.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:05:49.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:48 smithi167 ceph-mon[47728]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:49.471 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T19:05:49.471 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:49.471 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:49.471 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89097]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:49.472 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T19:05:49.472 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:49.472 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:49.472 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:05:49 smithi131 sudo[89100]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:49 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1824856216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:49 smithi131 ceph-mon[78559]: mgrmap e16: x(active, since 118s), standbys: z, y 2024-03-22T19:05:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:50.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:49 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1824856216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:50.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:49 smithi086 ceph-mon[38032]: mgrmap e16: x(active, since 118s), standbys: z, y 2024-03-22T19:05:50.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:50.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:50.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:49 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1824856216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:05:50.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:49 smithi167 ceph-mon[47728]: mgrmap e16: x(active, since 118s), standbys: z, y 2024-03-22T19:05:50.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:50.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58913]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58913]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58913]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58916]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58916]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58916]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:50.080 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:05:49 smithi167 sudo[58916]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:50.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:50 smithi086 sudo[63976]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T19:05:50.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:50 smithi086 sudo[63976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:50.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:50 smithi086 sudo[63976]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:50.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:50 smithi086 sudo[63976]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:51.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:51.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 ceph-mon[78559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:51.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 ceph-mon[78559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 sudo[64167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T19:05:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 sudo[64167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 sudo[64167]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 sudo[64167]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 ceph-mon[38032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:51 smithi086 ceph-mon[38032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58919]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58919]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58919]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58919]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 sudo[58922]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 ceph-mon[47728]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:51.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:51 smithi167 ceph-mon[47728]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:52.058 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 sudo[89103]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 sudo[89103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 sudo[89103]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:51 smithi131 sudo[89103]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 sudo[89106]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 sudo[89106]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 sudo[89106]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T19:05:52.059 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 sudo[89106]: pam_unix(sudo:session): session closed for user root 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: pgmap v71: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:52 smithi086 ceph-mon[38032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:52.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: pgmap v71: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:52.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:52.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:52 smithi167 ceph-mon[47728]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:52.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: pgmap v71: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:52.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:05:52.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:52 smithi131 ceph-mon[78559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T19:05:53.204 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:53 smithi086 ceph-mon[38032]: pgmap v72: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:53.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:53 smithi167 ceph-mon[47728]: pgmap v72: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:53.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:53 smithi131 ceph-mon[78559]: pgmap v72: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:05:55.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:55 smithi086 ceph-mon[38032]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 30 op/s 2024-03-22T19:05:55.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T19:05:55.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:55.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:55 smithi086 ceph-mon[38032]: Deploying daemon osd.3 on smithi086 2024-03-22T19:05:55.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:55 smithi131 ceph-mon[78559]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 30 op/s 2024-03-22T19:05:55.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T19:05:55.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:55.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:55 smithi131 ceph-mon[78559]: Deploying daemon osd.3 on smithi086 2024-03-22T19:05:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:55 smithi167 ceph-mon[47728]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 30 op/s 2024-03-22T19:05:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T19:05:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:05:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:55 smithi167 ceph-mon[47728]: Deploying daemon osd.3 on smithi086 2024-03-22T19:05:57.248 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:57 smithi086 ceph-mon[38032]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-22T19:05:57.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:57 smithi167 ceph-mon[47728]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-22T19:05:57.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:57 smithi131 ceph-mon[78559]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-22T19:05:59.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:05:59 smithi086 ceph-mon[38032]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-22T19:05:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:05:59 smithi131 ceph-mon[78559]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-22T19:05:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:05:59 smithi167 ceph-mon[47728]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-22T19:06:00.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:00.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:00.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:00.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:01.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:01 smithi086 ceph-mon[38032]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-22T19:06:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:01 smithi167 ceph-mon[47728]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-22T19:06:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:01 smithi131 ceph-mon[78559]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-22T19:06:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:02.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.550+0000 7f7853e00700 1 -- 172.21.15.86:0/1390397954 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f785c0611f0 con 0x7f78400a0a80 2024-03-22T19:06:02.552 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-03-22T19:06:02.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.552+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78400a0a80 msgr2=0x7f78400a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:02.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78400a0a80 0x7f78400a2f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f784800b5c0 tx=0x7f7848003790 comp rx=0 tx=0).stop 2024-03-22T19:06:02.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 msgr2=0x7f785c198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:02.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 0x7f785c198f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f784c00d980 tx=0x7f784c00dc90 comp rx=0 tx=0).stop 2024-03-22T19:06:02.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 shutdown_connections 2024-03-22T19:06:02.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78400a0a80 0x7f78400a2f40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:02.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f785c106b20 0x7f785c1a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:02.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f785c1018c0 0x7f785c198f90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:02.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 --2- 172.21.15.86:0/1390397954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f785c1041f0 0x7f785c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:02.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 >> 172.21.15.86:0/1390397954 conn(0x7f785c0fb270 msgr2=0x7f785c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:02.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 shutdown_connections 2024-03-22T19:06:02.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:02.553+0000 7f7862400700 1 -- 172.21.15.86:0/1390397954 wait complete. 2024-03-22T19:06:02.951 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.3.service 2024-03-22T19:06:02.954 INFO:tasks.cephadm:Deploying osd.4 on smithi131 with /dev/vg_nvme/lv_3... 2024-03-22T19:06:02.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T19:06:03.028 INFO:journalctl@ceph.osd.3.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:06:03.187 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:06:03.604 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:03 smithi131 ceph-mon[78559]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-22T19:06:03.604 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:03.604 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:03.604 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:03.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:03 smithi086 ceph-mon[38032]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-22T19:06:03.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:03.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:03.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:03.664 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:06:03 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3[65372]: 2024-03-22T19:06:03.535+0000 7fcb9558b640 -1 osd.3 0 log_to_monitors true 2024-03-22T19:06:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:03 smithi167 ceph-mon[47728]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-22T19:06:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T19:06:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T19:06:04.474 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-03-22T19:06:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-03-22T19:06:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011165 s, 939 MB/s 2024-03-22T19:06:04.475 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 2024-03-22T19:06:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-03-22T19:06:04.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:04 smithi086 ceph-mon[38032]: from='osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T19:06:04.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:04 smithi086 ceph-mon[38032]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T19:06:04.782 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:04 smithi131 ceph-mon[78559]: from='osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T19:06:04.783 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:04 smithi131 ceph-mon[78559]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T19:06:04.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-03-22T19:06:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:04 smithi167 ceph-mon[47728]: from='osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T19:06:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:04 smithi167 ceph-mon[47728]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T19:06:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:06:05.758 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3[65372]: 2024-03-22T19:06:05.528+0000 7fcb89c00700 -1 osd.3 0 waiting for initial osdmap 2024-03-22T19:06:05.759 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3[65372]: 2024-03-22T19:06:05.534+0000 7fcb80e00700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:06:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-mon[38032]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-22T19:06:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-mon[38032]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T19:06:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-mon[38032]: osdmap e22: 4 total, 3 up, 4 in 2024-03-22T19:06:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-mon[38032]: from='osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:06:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:05 smithi086 ceph-mon[38032]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:06:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:05 smithi167 ceph-mon[47728]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-22T19:06:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:05 smithi167 ceph-mon[47728]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T19:06:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:05 smithi167 ceph-mon[47728]: osdmap e22: 4 total, 3 up, 4 in 2024-03-22T19:06:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:05 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:05 smithi167 ceph-mon[47728]: from='osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:06:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:05 smithi167 ceph-mon[47728]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:06:05.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:05 smithi131 ceph-mon[78559]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-22T19:06:05.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:05 smithi131 ceph-mon[78559]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T19:06:05.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:05 smithi131 ceph-mon[78559]: osdmap e22: 4 total, 3 up, 4 in 2024-03-22T19:06:05.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:05 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:05.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:05 smithi131 ceph-mon[78559]: from='osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:06:05.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:05 smithi131 ceph-mon[78559]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:06:06.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 -- 172.21.15.131:0/942338262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90101fc0 msgr2=0x7f0f90102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:06.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 --2- 172.21.15.131:0/942338262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90101fc0 0x7f0f90102420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f800099c0 tx=0x7f0f80009cd0 comp rx=0 tx=0).stop 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 -- 172.21.15.131:0/942338262 shutdown_connections 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 --2- 172.21.15.131:0/942338262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f90102960 0x7f0f90109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 --2- 172.21.15.131:0/942338262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f90100dc0 0x7f0f901011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 --2- 172.21.15.131:0/942338262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90101fc0 0x7f0f90102420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 -- 172.21.15.131:0/942338262 >> 172.21.15.131:0/942338262 conn(0x7f0f900fc360 msgr2=0x7f0f900fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 -- 172.21.15.131:0/942338262 shutdown_connections 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 -- 172.21.15.131:0/942338262 wait complete. 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 Processor -- start 2024-03-22T19:06:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 -- start start 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 0x7f0f90196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.007+0000 7f0f97b37700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f90101fc0 0x7f0f90197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f97b37700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f90102960 0x7f0f9019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f97b37700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019e8e0 con 0x7f0f90102960 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f97b37700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019ea20 con 0x7f0f90100dc0 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f97b37700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019ebe0 con 0x7f0f90101fc0 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 0x7f0f90196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 0x7f0f90196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53972/0 (socket says 172.21.15.131:53972) 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 -- 172.21.15.131:0/1163046447 learned_addr learned my addr 172.21.15.131:0/1163046447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f960d4700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f90102960 0x7f0f9019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f950d2700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f90101fc0 0x7f0f90197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 -- 172.21.15.131:0/1163046447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f90101fc0 msgr2=0x7f0f90197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f90101fc0 0x7f0f90197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 -- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f90102960 msgr2=0x7f0f9019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f90102960 0x7f0f9019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 -- 172.21.15.131:0/1163046447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800095f0 con 0x7f0f90100dc0 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f958d3700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 0x7f0f90196de0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00b500 tx=0x7f0f8c00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c011830 con 0x7f0f90100dc0 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f8c011e50 con 0x7f0f90100dc0 2024-03-22T19:06:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9019eec0 con 0x7f0f90100dc0 2024-03-22T19:06:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00f2e0 con 0x7f0f90100dc0 2024-03-22T19:06:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.008+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9019f2e0 con 0x7f0f90100dc0 2024-03-22T19:06:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.009+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f90108290 con 0x7f0f90100dc0 2024-03-22T19:06:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.010+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0f8c010580 con 0x7f0f90100dc0 2024-03-22T19:06:06.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.011+0000 7f0f86ffd700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f7c0a0b10 0x7f0f7c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:06.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.011+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f0f8c0ca7f0 con 0x7f0f90100dc0 2024-03-22T19:06:06.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.011+0000 7f0f950d2700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f7c0a0b10 0x7f0f7c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:06.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.011+0000 7f0f950d2700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f7c0a0b10 0x7f0f7c0a2fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f80000c00 tx=0x7f0f80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:06:06.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.015+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0f8c095460 con 0x7f0f90100dc0 2024-03-22T19:06:06.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:06.218+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f900611f0 con 0x7f0f7c0a0b10 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] boot 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: osdmap e24: 4 total, 4 up, 4 in 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:06.929 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] boot 2024-03-22T19:06:06.930 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: osdmap e24: 4 total, 4 up, 4 in 2024-03-22T19:06:06.930 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:06.930 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:06.930 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:06.930 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:07.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: osd.3 [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] boot 2024-03-22T19:06:07.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: osdmap e24: 4 total, 4 up, 4 in 2024-03-22T19:06:07.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T19:06:07.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:07.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:07.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:07.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:06:07.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:06:07.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi086 2024-03-22T19:06:07.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi086 to 8635M 2024-03-22T19:06:07.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:06:07.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:07.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:07 smithi131 ceph-mon[78559]: osdmap e25: 4 total, 4 up, 4 in 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi086 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi086 to 8635M 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:07 smithi167 ceph-mon[47728]: osdmap e25: 4 total, 4 up, 4 in 2024-03-22T19:06:08.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:06:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:06:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi086 2024-03-22T19:06:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi086 to 8635M 2024-03-22T19:06:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-22T19:06:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:07 smithi086 ceph-mon[38032]: osdmap e25: 4 total, 4 up, 4 in 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/65611309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]: dispatch 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]: dispatch 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]': finished 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: osdmap e27: 5 total, 4 up, 5 in 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:09.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:09.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:09 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/1893671195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:09.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T19:06:09.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/65611309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]: dispatch 2024-03-22T19:06:09.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]: dispatch 2024-03-22T19:06:09.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]': finished 2024-03-22T19:06:09.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: osdmap e27: 5 total, 4 up, 5 in 2024-03-22T19:06:09.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:09.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:09.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:09 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/1893671195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:09.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/65611309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]: dispatch 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]: dispatch 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58f48274-3606-4e03-bd1c-1f58c186882d"}]': finished 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: osdmap e27: 5 total, 4 up, 5 in 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:09.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:09 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/1893671195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:10.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:10 smithi131 ceph-mon[78559]: osdmap e28: 5 total, 4 up, 5 in 2024-03-22T19:06:10.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:10.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:10 smithi086 ceph-mon[38032]: osdmap e28: 5 total, 4 up, 5 in 2024-03-22T19:06:10.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:10 smithi167 ceph-mon[47728]: osdmap e28: 5 total, 4 up, 5 in 2024-03-22T19:06:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:11.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:11 smithi131 ceph-mon[78559]: osdmap e29: 5 total, 4 up, 5 in 2024-03-22T19:06:11.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:11 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:11.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:11 smithi131 ceph-mon[78559]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 189 KiB/s, 0 objects/s recovering 2024-03-22T19:06:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:11 smithi086 ceph-mon[38032]: osdmap e29: 5 total, 4 up, 5 in 2024-03-22T19:06:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:11 smithi086 ceph-mon[38032]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 189 KiB/s, 0 objects/s recovering 2024-03-22T19:06:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:11 smithi167 ceph-mon[47728]: osdmap e29: 5 total, 4 up, 5 in 2024-03-22T19:06:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:11 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:11 smithi167 ceph-mon[47728]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 189 KiB/s, 0 objects/s recovering 2024-03-22T19:06:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:13 smithi131 ceph-mon[78559]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-03-22T19:06:13.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:13 smithi086 ceph-mon[38032]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-03-22T19:06:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:13 smithi167 ceph-mon[47728]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-03-22T19:06:14.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T19:06:14.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:14.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T19:06:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:14.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T19:06:14.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:15.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:15 smithi131 ceph-mon[78559]: Deploying daemon osd.4 on smithi131 2024-03-22T19:06:15.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:15 smithi131 ceph-mon[78559]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-22T19:06:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:15 smithi086 ceph-mon[38032]: Deploying daemon osd.4 on smithi131 2024-03-22T19:06:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:15 smithi086 ceph-mon[38032]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-22T19:06:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:15 smithi167 ceph-mon[47728]: Deploying daemon osd.4 on smithi131 2024-03-22T19:06:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:15 smithi167 ceph-mon[47728]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-22T19:06:17.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:17 smithi131 ceph-mon[78559]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T19:06:17.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:17 smithi167 ceph-mon[47728]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T19:06:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:17 smithi086 ceph-mon[38032]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T19:06:18.686 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:18.686 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:18.686 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:19.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:19 smithi167 ceph-mon[47728]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:19 smithi131 ceph-mon[78559]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:19.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.830 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:19.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:19.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:19.921 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 4 on host 'smithi131' 2024-03-22T19:06:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.920+0000 7f0f86ffd700 1 -- 172.21.15.131:0/1163046447 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f900611f0 con 0x7f0f7c0a0b10 2024-03-22T19:06:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f7c0a0b10 msgr2=0x7f0f7c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f7c0a0b10 0x7f0f7c0a2fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f80000c00 tx=0x7f0f80005fb0 comp rx=0 tx=0).stop 2024-03-22T19:06:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 msgr2=0x7f0f90196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 0x7f0f90196de0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00b500 tx=0x7f0f8c00b8c0 comp rx=0 tx=0).stop 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 shutdown_connections 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f7c0a0b10 0x7f0f7c0a2fd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f90102960 0x7f0f9019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f90100dc0 0x7f0f90196de0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 --2- 172.21.15.131:0/1163046447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f90101fc0 0x7f0f90197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 >> 172.21.15.131:0/1163046447 conn(0x7f0f900fc360 msgr2=0x7f0f901053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 shutdown_connections 2024-03-22T19:06:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:06:19.922+0000 7f0f97b37700 1 -- 172.21.15.131:0/1163046447 wait complete. 2024-03-22T19:06:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:19 smithi086 ceph-mon[38032]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:20.302 DEBUG:teuthology.orchestra.run.smithi131:osd.4> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.4.service 2024-03-22T19:06:20.305 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-03-22T19:06:20.305 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T19:06:20.336 INFO:journalctl@ceph.osd.4.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:06:20.539 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:06:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:20 smithi167 ceph-mon[47728]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:21.090 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:21.090 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:21.090 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:21.090 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:20 smithi131 ceph-mon[78559]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:21.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:21.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:21.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:21.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:20 smithi086 ceph-mon[38032]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:21.944 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T19:06:21.944 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T19:06:21.944 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-03-22T19:06:21.944 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-03-22T19:06:21.944 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112675 s, 931 MB/s 2024-03-22T19:06:21.944 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-03-22T19:06:21.971 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:06:21 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4[92175]: 2024-03-22T19:06:21.555+0000 7efedbd87640 -1 osd.4 0 log_to_monitors true 2024-03-22T19:06:22.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:22 smithi167 ceph-mon[47728]: from='osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T19:06:22.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:22 smithi167 ceph-mon[47728]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T19:06:22.313 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-03-22T19:06:22.413 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:22 smithi131 ceph-mon[78559]: from='osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T19:06:22.413 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:22 smithi131 ceph-mon[78559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T19:06:22.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:22 smithi086 ceph-mon[38032]: from='osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T19:06:22.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:22 smithi086 ceph-mon[38032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T19:06:22.533 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: osdmap e30: 5 total, 4 up, 5 in 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:23.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi131 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi131 to 8556M 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:23.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.471 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4[92175]: 2024-03-22T19:06:23.078+0000 7efed13eb700 -1 osd.4 0 waiting for initial osdmap 2024-03-22T19:06:23.472 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4[92175]: 2024-03-22T19:06:23.084+0000 7efecb1dc700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: osdmap e30: 5 total, 4 up, 5 in 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi131 2024-03-22T19:06:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi131 to 8556M 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:23.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T19:06:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: osdmap e30: 5 total, 4 up, 5 in 2024-03-22T19:06:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi131 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi131 to 8556M 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 -- 172.21.15.167:0/4265064045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 msgr2=0x7f0d4c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 --2- 172.21.15.167:0/4265064045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c108120 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400b0a0 tx=0x7f0d4400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 -- 172.21.15.167:0/4265064045 shutdown_connections 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 --2- 172.21.15.167:0/4265064045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c108120 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 --2- 172.21.15.167:0/4265064045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c0ffcd0 0x7f0d4c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 --2- 172.21.15.167:0/4265064045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d4c100ed0 0x7f0d4c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 -- 172.21.15.167:0/4265064045 >> 172.21.15.167:0/4265064045 conn(0x7f0d4c0fb270 msgr2=0x7f0d4c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 -- 172.21.15.167:0/4265064045 shutdown_connections 2024-03-22T19:06:23.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.553+0000 7f0d544f0700 1 -- 172.21.15.167:0/4265064045 wait complete. 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 Processor -- start 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 -- start start 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d4c0ffcd0 0x7f0d4c196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c100ed0 0x7f0d4c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c19e960 con 0x7f0d4c100ed0 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c19eaa0 con 0x7f0d4c0ffcd0 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d544f0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c19ec30 con 0x7f0d4c101870 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51456/0 (socket says 172.21.15.167:51456) 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 -- 172.21.15.167:0/3897664663 learned_addr learned my addr 172.21.15.167:0/3897664663 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d5228c700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d4c0ffcd0 0x7f0d4c196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 -- 172.21.15.167:0/3897664663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d4c0ffcd0 msgr2=0x7f0d4c196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:23.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d4c0ffcd0 0x7f0d4c196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 -- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c100ed0 msgr2=0x7f0d4c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c100ed0 0x7f0d4c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.554+0000 7f0d52a8d700 1 -- 172.21.15.167:0/3897664663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44009d00 con 0x7f0d4c101870 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.555+0000 7f0d52a8d700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c19e390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d44004d30 tx=0x7f0d44004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.555+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400e040 con 0x7f0d4c101870 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.555+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d4400ce50 con 0x7f0d4c101870 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.555+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c19eeb0 con 0x7f0d4c101870 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.555+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4401bb60 con 0x7f0d4c101870 2024-03-22T19:06:23.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.555+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c19f2c0 con 0x7f0d4c101870 2024-03-22T19:06:23.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.556+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c04eaf0 con 0x7f0d4c101870 2024-03-22T19:06:23.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.557+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0d4401bcc0 con 0x7f0d4c101870 2024-03-22T19:06:23.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.558+0000 7f0d437fe700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d380a0a90 0x7f0d380a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:23.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.558+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f0d440cd0e0 con 0x7f0d4c101870 2024-03-22T19:06:23.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.559+0000 7f0d5228c700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d380a0a90 0x7f0d380a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:23.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.559+0000 7f0d5228c700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d380a0a90 0x7f0d380a2f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c100d30 tx=0x7f0d48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:06:23.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.562+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0d44097c30 con 0x7f0d4c101870 2024-03-22T19:06:23.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:23.760+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d4c04ea80 con 0x7f0d380a0a90 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:24.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:06:24.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T19:06:24.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:24.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:24.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:24.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:24.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:06:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T19:06:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:24.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:24.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:24.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:24.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:06:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:06:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:25.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] boot 2024-03-22T19:06:25.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: osdmap e32: 5 total, 5 up, 5 in 2024-03-22T19:06:25.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:25.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:25 smithi131 ceph-mon[78559]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:06:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:06:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] boot 2024-03-22T19:06:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: osdmap e32: 5 total, 5 up, 5 in 2024-03-22T19:06:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:25 smithi086 ceph-mon[38032]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: osd.4 [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] boot 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: osdmap e32: 5 total, 5 up, 5 in 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T19:06:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:25 smithi167 ceph-mon[47728]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-03-22T19:06:26.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:26 smithi131 ceph-mon[78559]: osdmap e33: 5 total, 5 up, 5 in 2024-03-22T19:06:26.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:26 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/292253001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]: dispatch 2024-03-22T19:06:26.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:26 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]: dispatch 2024-03-22T19:06:26.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:26 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]': finished 2024-03-22T19:06:26.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:26 smithi131 ceph-mon[78559]: osdmap e34: 6 total, 5 up, 6 in 2024-03-22T19:06:26.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:26 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:26.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:26 smithi086 ceph-mon[38032]: osdmap e33: 5 total, 5 up, 5 in 2024-03-22T19:06:26.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:26 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/292253001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]: dispatch 2024-03-22T19:06:26.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:26 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]: dispatch 2024-03-22T19:06:26.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:26 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]': finished 2024-03-22T19:06:26.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:26 smithi086 ceph-mon[38032]: osdmap e34: 6 total, 5 up, 6 in 2024-03-22T19:06:26.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:26 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:26 smithi167 ceph-mon[47728]: osdmap e33: 5 total, 5 up, 5 in 2024-03-22T19:06:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:26 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/292253001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]: dispatch 2024-03-22T19:06:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:26 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]: dispatch 2024-03-22T19:06:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:26 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4140705c-b24d-488d-8db0-7f6093013c8a"}]': finished 2024-03-22T19:06:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:26 smithi167 ceph-mon[47728]: osdmap e34: 6 total, 5 up, 6 in 2024-03-22T19:06:26.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:26 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:27 smithi131 ceph-mon[78559]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:27 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/724996650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:27 smithi086 ceph-mon[38032]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:27 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/724996650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:27 smithi167 ceph-mon[47728]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:27 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/724996650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:29.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:29 smithi131 ceph-mon[78559]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:29.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:29 smithi167 ceph-mon[47728]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:29.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:29 smithi086 ceph-mon[38032]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:31.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:31 smithi131 ceph-mon[78559]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:31.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:31 smithi086 ceph-mon[38032]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:31.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:31 smithi167 ceph-mon[47728]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:32.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T19:06:32.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:32.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T19:06:32.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:32.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T19:06:32.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:33.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:33 smithi131 ceph-mon[78559]: Deploying daemon osd.5 on smithi167 2024-03-22T19:06:33.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:33 smithi131 ceph-mon[78559]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:33.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:33 smithi167 ceph-mon[47728]: Deploying daemon osd.5 on smithi167 2024-03-22T19:06:33.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:33 smithi167 ceph-mon[47728]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:33.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:33 smithi086 ceph-mon[38032]: Deploying daemon osd.5 on smithi167 2024-03-22T19:06:33.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:33 smithi086 ceph-mon[38032]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:35.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:35 smithi131 ceph-mon[78559]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:35.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:35 smithi086 ceph-mon[38032]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:35.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:35 smithi167 ceph-mon[47728]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:36.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:36 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:36.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:36 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:36.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:36 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:36.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:36 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:36.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:36.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:36.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:36 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:37 smithi167 ceph-mon[47728]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:37.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:37.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.718+0000 7f0d437fe700 1 -- 172.21.15.167:0/3897664663 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0d4c04ea80 con 0x7f0d380a0a90 2024-03-22T19:06:37.721 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-03-22T19:06:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:37 smithi131 ceph-mon[78559]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d380a0a90 msgr2=0x7f0d380a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d380a0a90 0x7f0d380a2f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c100d30 tx=0x7f0d48009190 comp rx=0 tx=0).stop 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 msgr2=0x7f0d4c19e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c19e390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d44004d30 tx=0x7f0d44004d60 comp rx=0 tx=0).stop 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 shutdown_connections 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d380a0a90 0x7f0d380a2f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4c101870 0x7f0d4c19e390 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d4c0ffcd0 0x7f0d4c196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 --2- 172.21.15.167:0/3897664663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c100ed0 0x7f0d4c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 >> 172.21.15.167:0/3897664663 conn(0x7f0d4c0fb270 msgr2=0x7f0d4c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 shutdown_connections 2024-03-22T19:06:37.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:06:37.721+0000 7f0d544f0700 1 -- 172.21.15.167:0/3897664663 wait complete. 2024-03-22T19:06:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:37 smithi086 ceph-mon[38032]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:37.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:38.212 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.5.service 2024-03-22T19:06:38.214 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-03-22T19:06:38.214 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T19:06:38.245 INFO:journalctl@ceph.osd.5.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:06:38.573 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:06:38.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:38.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:38.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:38.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:38 smithi167 ceph-mon[47728]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:38.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:38.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:38.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:38.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:38 smithi131 ceph-mon[78559]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:39.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:39.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:39.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:38 smithi086 ceph-mon[38032]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:39.707 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:06:39 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5[62105]: 2024-03-22T19:06:39.414+0000 7ffa882bf640 -1 osd.5 0 log_to_monitors true 2024-03-22T19:06:39.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:39 smithi086 ceph-mon[38032]: from='osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T19:06:39.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:39 smithi086 ceph-mon[38032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T19:06:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:39 smithi131 ceph-mon[78559]: from='osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T19:06:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:39 smithi131 ceph-mon[78559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T19:06:40.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:39 smithi167 ceph-mon[47728]: from='osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T19:06:40.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:39 smithi167 ceph-mon[47728]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T19:06:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T19:06:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T19:06:40.305 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-22T19:06:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-22T19:06:40.305 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0429845 s, 244 MB/s 2024-03-22T19:06:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-22T19:06:40.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-03-22T19:06:41.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: osdmap e35: 6 total, 5 up, 6 in 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi167 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi167 to 8557M 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:41.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:41.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:40 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:06:41.078 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5[62105]: 2024-03-22T19:06:40.728+0000 7ffa7e124700 -1 osd.5 0 waiting for initial osdmap 2024-03-22T19:06:41.078 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5[62105]: 2024-03-22T19:06:40.735+0000 7ffa77714700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:06:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T19:06:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: osdmap e35: 6 total, 5 up, 6 in 2024-03-22T19:06:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:06:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:06:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi167 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi167 to 8557M 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:40 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T19:06:41.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: osdmap e35: 6 total, 5 up, 6 in 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi167 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi167 to 8557M 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:41.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:41.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:40 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:42.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:06:42.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T19:06:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] boot 2024-03-22T19:06:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: osdmap e37: 6 total, 6 up, 6 in 2024-03-22T19:06:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:06:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T19:06:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] boot 2024-03-22T19:06:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: osdmap e37: 6 total, 6 up, 6 in 2024-03-22T19:06:42.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:06:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T19:06:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: osd.5 [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] boot 2024-03-22T19:06:42.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: osdmap e37: 6 total, 6 up, 6 in 2024-03-22T19:06:42.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T19:06:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 -- 172.21.15.86:0/2364636174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c101fe0 msgr2=0x7f410c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 --2- 172.21.15.86:0/2364636174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c101fe0 0x7f410c102440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f40f80099c0 tx=0x7f40f8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:06:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 -- 172.21.15.86:0/2364636174 shutdown_connections 2024-03-22T19:06:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 --2- 172.21.15.86:0/2364636174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c102980 0x7f410c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 --2- 172.21.15.86:0/2364636174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c100de0 0x7f410c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 --2- 172.21.15.86:0/2364636174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c101fe0 0x7f410c102440 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.264+0000 7f4111800700 1 -- 172.21.15.86:0/2364636174 >> 172.21.15.86:0/2364636174 conn(0x7f410c0fc380 msgr2=0x7f410c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.265+0000 7f4111800700 1 -- 172.21.15.86:0/2364636174 shutdown_connections 2024-03-22T19:06:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.265+0000 7f4111800700 1 -- 172.21.15.86:0/2364636174 wait complete. 2024-03-22T19:06:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 Processor -- start 2024-03-22T19:06:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 -- start start 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 0x7f410c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c101fe0 0x7f410c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c102980 0x7f410c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19e830 con 0x7f410c100de0 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19e970 con 0x7f410c102980 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f4111800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19eb30 con 0x7f410c101fe0 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f410aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 0x7f410c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f410a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c101fe0 0x7f410c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f410aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 0x7f410c196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38776/0 (socket says 172.21.15.86:38776) 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f410a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c101fe0 0x7f410c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:38530/0 (socket says 172.21.15.86:38530) 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.266+0000 7f410aa00700 1 -- 172.21.15.86:0/1707723719 learned_addr learned my addr 172.21.15.86:0/1707723719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:06:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410b400700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c102980 0x7f410c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410aa00700 1 -- 172.21.15.86:0/1707723719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c101fe0 msgr2=0x7f410c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410aa00700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c101fe0 0x7f410c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410aa00700 1 -- 172.21.15.86:0/1707723719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c102980 msgr2=0x7f410c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410aa00700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c102980 0x7f410c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410aa00700 1 -- 172.21.15.86:0/1707723719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80095f0 con 0x7f410c100de0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f410aa00700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 0x7f410c196d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00d980 tx=0x7f40fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc004020 con 0x7f410c100de0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40fc00f690 con 0x7f410c100de0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c073040 con 0x7f410c100de0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc003a60 con 0x7f410c100de0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.267+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c073590 con 0x7f410c100de0 2024-03-22T19:06:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.268+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c108260 con 0x7f410c100de0 2024-03-22T19:06:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.270+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f40fc00fb00 con 0x7f410c100de0 2024-03-22T19:06:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.271+0000 7f4103400700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f40f00a0b10 0x7f40f00a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:06:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.272+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f40fc0cb0e0 con 0x7f410c100de0 2024-03-22T19:06:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.272+0000 7f410a000700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f40f00a0b10 0x7f40f00a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:06:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.272+0000 7f410a000700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f40f00a0b10 0x7f40f00a2fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006010 tx=0x7f40f800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:06:42.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.275+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f40fc095930 con 0x7f410c100de0 2024-03-22T19:06:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:42.482+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f410c0611f0 con 0x7f40f00a0b10 2024-03-22T19:06:43.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:06:43.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:06:43.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: osdmap e38: 6 total, 6 up, 6 in 2024-03-22T19:06:43.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:06:43.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:43.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:43.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:43.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: osdmap e38: 6 total, 6 up, 6 in 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:43.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:43.407 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:43.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:06:43.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:06:43.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: osdmap e38: 6 total, 6 up, 6 in 2024-03-22T19:06:43.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:06:43.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:06:43.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:06:43.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:06:43.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:44.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:44 smithi086 ceph-mon[38032]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T19:06:44.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:44 smithi167 ceph-mon[47728]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T19:06:44.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:44 smithi131 ceph-mon[78559]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T19:06:45.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:45 smithi131 ceph-mon[78559]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T19:06:45.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:45 smithi131 ceph-mon[78559]: pgmap v117: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:45.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:45 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2506448230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f54546-8e74-4eb8-9489-9b5155f0501e"}]: dispatch 2024-03-22T19:06:45.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:45 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2506448230' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f54546-8e74-4eb8-9489-9b5155f0501e"}]': finished 2024-03-22T19:06:45.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:45 smithi131 ceph-mon[78559]: osdmap e41: 7 total, 6 up, 7 in 2024-03-22T19:06:45.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:06:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:45 smithi086 ceph-mon[38032]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T19:06:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:45 smithi086 ceph-mon[38032]: pgmap v117: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:45 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2506448230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f54546-8e74-4eb8-9489-9b5155f0501e"}]: dispatch 2024-03-22T19:06:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:45 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2506448230' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f54546-8e74-4eb8-9489-9b5155f0501e"}]': finished 2024-03-22T19:06:45.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:45 smithi086 ceph-mon[38032]: osdmap e41: 7 total, 6 up, 7 in 2024-03-22T19:06:45.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:06:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:45 smithi167 ceph-mon[47728]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T19:06:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:45 smithi167 ceph-mon[47728]: pgmap v117: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:45 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2506448230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f54546-8e74-4eb8-9489-9b5155f0501e"}]: dispatch 2024-03-22T19:06:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:45 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2506448230' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f54546-8e74-4eb8-9489-9b5155f0501e"}]': finished 2024-03-22T19:06:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:45 smithi167 ceph-mon[47728]: osdmap e41: 7 total, 6 up, 7 in 2024-03-22T19:06:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:06:46.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:46 smithi131 ceph-mon[78559]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-22T19:06:46.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:46 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1252947421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:46.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:46 smithi086 ceph-mon[38032]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-22T19:06:46.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:46 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1252947421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:46 smithi167 ceph-mon[47728]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-22T19:06:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:46 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1252947421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:06:47.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:47 smithi131 ceph-mon[78559]: pgmap v119: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:47 smithi086 ceph-mon[38032]: pgmap v119: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:47.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:47 smithi167 ceph-mon[47728]: pgmap v119: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:49.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:49 smithi131 ceph-mon[78559]: pgmap v120: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:49.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:49 smithi086 ceph-mon[38032]: pgmap v120: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:49 smithi167 ceph-mon[47728]: pgmap v120: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-22T19:06:50.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:50 smithi086 ceph-mon[38032]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-22T19:06:50.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:50 smithi086 ceph-mon[38032]: Cluster is now healthy 2024-03-22T19:06:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:50 smithi167 ceph-mon[47728]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-22T19:06:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:50 smithi167 ceph-mon[47728]: Cluster is now healthy 2024-03-22T19:06:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:50 smithi131 ceph-mon[78559]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-22T19:06:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:50 smithi131 ceph-mon[78559]: Cluster is now healthy 2024-03-22T19:06:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:51 smithi086 ceph-mon[38032]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-22T19:06:51.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:51 smithi167 ceph-mon[47728]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-22T19:06:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:51 smithi131 ceph-mon[78559]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-22T19:06:52.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-22T19:06:52.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:52.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:52 smithi086 ceph-mon[38032]: Deploying daemon osd.6 on smithi086 2024-03-22T19:06:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-22T19:06:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:52 smithi167 ceph-mon[47728]: Deploying daemon osd.6 on smithi086 2024-03-22T19:06:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-22T19:06:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:52 smithi131 ceph-mon[78559]: Deploying daemon osd.6 on smithi086 2024-03-22T19:06:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:53 smithi086 ceph-mon[38032]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T19:06:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:53 smithi167 ceph-mon[47728]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T19:06:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:53 smithi131 ceph-mon[78559]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T19:06:55.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:55 smithi086 ceph-mon[38032]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-22T19:06:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:55 smithi167 ceph-mon[47728]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-22T19:06:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:55 smithi131 ceph-mon[78559]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-22T19:06:57.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:57 smithi086 ceph-mon[38032]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-22T19:06:57.558 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:57.558 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:57.558 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:57 smithi167 ceph-mon[47728]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-22T19:06:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:57.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:57 smithi131 ceph-mon[78559]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-22T19:06:57.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:57.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:57.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:59.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.184+0000 7f4103400700 1 -- 172.21.15.86:0/1707723719 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f410c0611f0 con 0x7f40f00a0b10 2024-03-22T19:06:59.187 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-03-22T19:06:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f40f00a0b10 msgr2=0x7f40f00a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f40f00a0b10 0x7f40f00a2fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006010 tx=0x7f40f800b540 comp rx=0 tx=0).stop 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 msgr2=0x7f410c196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 0x7f410c196d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00d980 tx=0x7f40fc00dc90 comp rx=0 tx=0).stop 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 shutdown_connections 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f40f00a0b10 0x7f40f00a2fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c102980 0x7f410c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410c100de0 0x7f410c196d30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 --2- 172.21.15.86:0/1707723719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f410c101fe0 0x7f410c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 >> 172.21.15.86:0/1707723719 conn(0x7f410c0fc380 msgr2=0x7f410c1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 shutdown_connections 2024-03-22T19:06:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:06:59.187+0000 7f4111800700 1 -- 172.21.15.86:0/1707723719 wait complete. 2024-03-22T19:06:59.604 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.6.service 2024-03-22T19:06:59.608 INFO:tasks.cephadm:Deploying osd.7 on smithi131 with /dev/vg_nvme/lv_2... 2024-03-22T19:06:59.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T19:06:59.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-22T19:06:59.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:59.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:59.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:59.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:06:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.679 INFO:journalctl@ceph.osd.6.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:06:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-22T19:06:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:06:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.842 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.963 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:06:59.964 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:06:59.964 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:06:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:01.009 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:07:00 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6[74030]: 2024-03-22T19:07:00.550+0000 7f546e0ee640 -1 osd.6 0 log_to_monitors true 2024-03-22T19:07:01.215 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T19:07:01.215 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T19:07:01.216 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-03-22T19:07:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-03-22T19:07:01.216 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153667 s, 682 MB/s 2024-03-22T19:07:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-03-22T19:07:01.566 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:01 smithi131 ceph-mon[78559]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-22T19:07:01.566 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:01 smithi131 ceph-mon[78559]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-22T19:07:01.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-03-22T19:07:01.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:01 smithi086 ceph-mon[38032]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-22T19:07:01.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:01 smithi086 ceph-mon[38032]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-22T19:07:01.803 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:07:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:01 smithi167 ceph-mon[47728]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-22T19:07:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:01 smithi167 ceph-mon[47728]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-22T19:07:02.758 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6[74030]: 2024-03-22T19:07:02.525+0000 7f5462600700 -1 osd.6 0 waiting for initial osdmap 2024-03-22T19:07:02.759 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6[74030]: 2024-03-22T19:07:02.541+0000 7f5459800700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: osdmap e42: 7 total, 6 up, 7 in 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 -- 172.21.15.131:0/3730451840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 msgr2=0x7f9ac41065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 --2- 172.21.15.131:0/3730451840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac41065e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009990 tx=0x7f9ab4009ca0 comp rx=0 tx=0).stop 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 -- 172.21.15.131:0/3730451840 shutdown_connections 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 --2- 172.21.15.131:0/3730451840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4106b20 0x7f9ac4108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 --2- 172.21.15.131:0/3730451840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac41018c0 0x7f9ac4103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 --2- 172.21.15.131:0/3730451840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac41065e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 -- 172.21.15.131:0/3730451840 >> 172.21.15.131:0/3730451840 conn(0x7f9ac40fb2b0 msgr2=0x7f9ac40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 -- 172.21.15.131:0/3730451840 shutdown_connections 2024-03-22T19:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.789+0000 7f9ac9990700 1 -- 172.21.15.131:0/3730451840 wait complete. 2024-03-22T19:07:02.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 Processor -- start 2024-03-22T19:07:02.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 -- start start 2024-03-22T19:07:02.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac41018c0 0x7f9ac41011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:02.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac40ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:02.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4106b20 0x7f9ac40ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4100550 con 0x7f9ac41018c0 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41006c0 con 0x7f9ac41041f0 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac9990700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41a0e40 con 0x7f9ac4106b20 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac40ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac40ff870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60750/0 (socket says 172.21.15.131:60750) 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.790+0000 7f9ac27fc700 1 -- 172.21.15.131:0/505740768 learned_addr learned my addr 172.21.15.131:0/505740768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac2ffd700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac41018c0 0x7f9ac41011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac37fe700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4106b20 0x7f9ac40ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac27fc700 1 -- 172.21.15.131:0/505740768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4106b20 msgr2=0x7f9ac40ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac27fc700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4106b20 0x7f9ac40ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac27fc700 1 -- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac41018c0 msgr2=0x7f9ac41011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac27fc700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac41018c0 0x7f9ac41011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:02.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac27fc700 1 -- 172.21.15.131:0/505740768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40095f0 con 0x7f9ac41041f0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac37fe700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4106b20 0x7f9ac40ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac2ffd700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac41018c0 0x7f9ac41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac27fc700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac40ff870 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab400f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab401d070 con 0x7f9ac41041f0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ab400fca0 con 0x7f9ac41041f0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac41a1070 con 0x7f9ac41041f0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4017810 con 0x7f9ac41041f0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.791+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac41a1480 con 0x7f9ac41041f0 2024-03-22T19:07:02.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.792+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40fce30 con 0x7f9ac41041f0 2024-03-22T19:07:02.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.793+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9ab4022ac0 con 0x7f9ac41041f0 2024-03-22T19:07:02.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.794+0000 7f9ac898e700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9ab00a0b10 0x7f9ab00a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:02.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.794+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f9ab40cc870 con 0x7f9ac41041f0 2024-03-22T19:07:02.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.794+0000 7f9ac2ffd700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9ab00a0b10 0x7f9ab00a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:02.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.795+0000 7f9ac2ffd700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9ab00a0b10 0x7f9ab00a2fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0099d0 tx=0x7f9aac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:02.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.798+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9ab4097060 con 0x7f9ac41041f0 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: osdmap e42: 7 total, 6 up, 7 in 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:02.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: osdmap e42: 7 total, 6 up, 7 in 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:02.998+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac40611f0 con 0x7f9ab00a0b10 2024-03-22T19:07:03.720 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:07:03.720 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi086 2024-03-22T19:07:03.720 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:03.720 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi086 to 5756M 2024-03-22T19:07:03.720 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:07:03.720 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: osdmap e43: 7 total, 6 up, 7 in 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi086 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi086 to 5756M 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: osdmap e43: 7 total, 6 up, 7 in 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:03.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:03.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:03.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:03.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:04.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi086 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi086 to 5756M 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: osdmap e43: 7 total, 6 up, 7 in 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:04.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:04.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:04.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:04.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] boot 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: osdmap e44: 7 total, 7 up, 7 in 2024-03-22T19:07:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:04 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] boot 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: osdmap e44: 7 total, 7 up, 7 in 2024-03-22T19:07:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:04 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:05.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:07:05.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:07:05.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:05.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:05.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: osd.6 [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] boot 2024-03-22T19:07:05.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: osdmap e44: 7 total, 7 up, 7 in 2024-03-22T19:07:05.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:04 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-22T19:07:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:07:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: osdmap e45: 7 total, 7 up, 7 in 2024-03-22T19:07:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/1837772324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]: dispatch 2024-03-22T19:07:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]: dispatch 2024-03-22T19:07:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]': finished 2024-03-22T19:07:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: osdmap e46: 8 total, 7 up, 8 in 2024-03-22T19:07:05.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:05.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:05 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/3311214165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:05.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: osdmap e45: 7 total, 7 up, 7 in 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/1837772324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]: dispatch 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]: dispatch 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]': finished 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: osdmap e46: 8 total, 7 up, 8 in 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:05 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/3311214165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T19:07:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: osdmap e45: 7 total, 7 up, 7 in 2024-03-22T19:07:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/1837772324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]: dispatch 2024-03-22T19:07:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]: dispatch 2024-03-22T19:07:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e490f98-55e1-44e0-8c95-234d0d2b05d5"}]': finished 2024-03-22T19:07:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: osdmap e46: 8 total, 7 up, 8 in 2024-03-22T19:07:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:05 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/3311214165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:07.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:07 smithi131 ceph-mon[78559]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:07.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:07 smithi086 ceph-mon[38032]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:07 smithi167 ceph-mon[47728]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:09.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:09 smithi131 ceph-mon[78559]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:09.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:09 smithi086 ceph-mon[38032]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:09 smithi167 ceph-mon[47728]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:11.302 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:11 smithi131 ceph-mon[78559]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:11.302 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:11 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-22T19:07:11.302 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:11 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:11 smithi086 ceph-mon[38032]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-22T19:07:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:11 smithi167 ceph-mon[47728]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:11 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-22T19:07:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:11 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:12.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:12 smithi131 ceph-mon[78559]: Deploying daemon osd.7 on smithi131 2024-03-22T19:07:12.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:12 smithi086 ceph-mon[38032]: Deploying daemon osd.7 on smithi131 2024-03-22T19:07:12.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:12 smithi167 ceph-mon[47728]: Deploying daemon osd.7 on smithi131 2024-03-22T19:07:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:13 smithi131 ceph-mon[78559]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:13.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:13 smithi086 ceph-mon[38032]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:13 smithi167 ceph-mon[47728]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:15 smithi131 ceph-mon[78559]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:15 smithi086 ceph-mon[38032]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:15 smithi167 ceph-mon[47728]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:16.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.552+0000 7f9ac898e700 1 -- 172.21.15.131:0/505740768 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9ac40611f0 con 0x7f9ab00a0b10 2024-03-22T19:07:16.553 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 7 on host 'smithi131' 2024-03-22T19:07:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9ab00a0b10 msgr2=0x7f9ab00a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9ab00a0b10 0x7f9ab00a2fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0099d0 tx=0x7f9aac008040 comp rx=0 tx=0).stop 2024-03-22T19:07:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 msgr2=0x7f9ac40ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac40ff870 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab400f7b0 comp rx=0 tx=0).stop 2024-03-22T19:07:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 shutdown_connections 2024-03-22T19:07:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9ab00a0b10 0x7f9ab00a2fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4106b20 0x7f9ac40ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac41018c0 0x7f9ac41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 --2- 172.21.15.131:0/505740768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac41041f0 0x7f9ac40ff870 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 >> 172.21.15.131:0/505740768 conn(0x7f9ac40fb2b0 msgr2=0x7f9ac4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 shutdown_connections 2024-03-22T19:07:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:16.555+0000 7f9ac9990700 1 -- 172.21.15.131:0/505740768 wait complete. 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.718 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.719 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:16.934 DEBUG:teuthology.orchestra.run.smithi131:osd.7> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.7.service 2024-03-22T19:07:16.936 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-03-22T19:07:16.936 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T19:07:16.991 INFO:journalctl@ceph.osd.7.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:07:17.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:17.182 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:07:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:17 smithi167 ceph-mon[47728]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:17.844 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:17 smithi131 ceph-mon[78559]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:18.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:17 smithi086 ceph-mon[38032]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:18.472 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:07:18 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7[100899]: 2024-03-22T19:07:18.136+0000 7f6d5c49c640 -1 osd.7 0 log_to_monitors true 2024-03-22T19:07:18.519 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T19:07:18.520 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T19:07:18.520 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-03-22T19:07:18.520 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-03-22T19:07:18.520 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148995 s, 704 MB/s 2024-03-22T19:07:18.520 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-03-22T19:07:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:18 smithi167 ceph-mon[47728]: from='osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-22T19:07:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:18 smithi167 ceph-mon[47728]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-22T19:07:18.964 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-03-22T19:07:18.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:18 smithi131 ceph-mon[78559]: from='osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-22T19:07:18.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:18 smithi131 ceph-mon[78559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-22T19:07:19.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:18 smithi086 ceph-mon[38032]: from='osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-22T19:07:19.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:18 smithi086 ceph-mon[38032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-22T19:07:19.181 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:07:19.971 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7[100899]: 2024-03-22T19:07:19.589+0000 7f6d51b00700 -1 osd.7 0 waiting for initial osdmap 2024-03-22T19:07:19.972 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7[100899]: 2024-03-22T19:07:19.601+0000 7f6d4b0f0700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: osdmap e47: 8 total, 7 up, 8 in 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:19.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-22T19:07:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: osdmap e47: 8 total, 7 up, 8 in 2024-03-22T19:07:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:19.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:20.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: osdmap e47: 8 total, 7 up, 8 in 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 -- 172.21.15.167:0/161654071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22340ffcd0 msgr2=0x7f22341000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 --2- 172.21.15.167:0/161654071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22340ffcd0 0x7f22341000f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2228009990 tx=0x7f2228009ca0 comp rx=0 tx=0).stop 2024-03-22T19:07:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 -- 172.21.15.167:0/161654071 shutdown_connections 2024-03-22T19:07:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 --2- 172.21.15.167:0/161654071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2234101870 0x7f2234108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:20.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 --2- 172.21.15.167:0/161654071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22340ffcd0 0x7f22341000f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:20.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 --2- 172.21.15.167:0/161654071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2234100ed0 0x7f2234101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:20.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 -- 172.21.15.167:0/161654071 >> 172.21.15.167:0/161654071 conn(0x7f22340fb270 msgr2=0x7f22340fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:20.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 -- 172.21.15.167:0/161654071 shutdown_connections 2024-03-22T19:07:20.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.203+0000 7f223a7e3700 1 -- 172.21.15.167:0/161654071 wait complete. 2024-03-22T19:07:20.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 Processor -- start 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 -- start start 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22340ffcd0 0x7f2234103b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2234100ed0 0x7f22341022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 0x7f22341027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234102ea0 con 0x7f2234100ed0 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234103010 con 0x7f22340ffcd0 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.204+0000 7f223a7e3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341a0e00 con 0x7f2234101870 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 0x7f22341027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2233fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22340ffcd0 0x7f2234103b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 0x7f22341027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57328/0 (socket says 172.21.15.167:57328) 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 -- 172.21.15.167:0/3964341911 learned_addr learned my addr 172.21.15.167:0/3964341911 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2233fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22340ffcd0 0x7f2234103b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.167:43164/0 (socket says 172.21.15.167:43164) 2024-03-22T19:07:20.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 -- 172.21.15.167:0/3964341911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22340ffcd0 msgr2=0x7f2234103b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22340ffcd0 0x7f2234103b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 -- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2234100ed0 msgr2=0x7f22341022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2234100ed0 0x7f22341022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 -- 172.21.15.167:0/3964341911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22280095f0 con 0x7f2234101870 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f2238d80700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 0x7f22341027f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b970 tx=0x7f222c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00c930 con 0x7f2234101870 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22341a0fa0 con 0x7f2234101870 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f222c014440 con 0x7f2234101870 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c0075d0 con 0x7f2234101870 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.205+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341a13c0 con 0x7f2234101870 2024-03-22T19:07:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.206+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223404ea80 con 0x7f2234101870 2024-03-22T19:07:20.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.207+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f222c0145b0 con 0x7f2234101870 2024-03-22T19:07:20.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.208+0000 7f22317fa700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f221c0a0a90 0x7f221c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:20.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.208+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f222c006e40 con 0x7f2234101870 2024-03-22T19:07:20.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.208+0000 7f2233fff700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f221c0a0a90 0x7f221c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:20.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.209+0000 7f2233fff700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f221c0a0a90 0x7f221c0a2f50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2228003790 tx=0x7f2228005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.212+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f222c0950d0 con 0x7f2234101870 2024-03-22T19:07:20.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:20.414+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2234068e30 con 0x7f221c0a0a90 2024-03-22T19:07:20.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi131 2024-03-22T19:07:20.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi131 to 5704M 2024-03-22T19:07:20.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:07:20.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: osdmap e48: 8 total, 7 up, 8 in 2024-03-22T19:07:20.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:20.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:20.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:20.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:20.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi131 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi131 to 5704M 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: osdmap e48: 8 total, 7 up, 8 in 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:20.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi131 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi131 to 5704M 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: osdmap e48: 8 total, 7 up, 8 in 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:21.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:21.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:07:21.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:07:21.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] boot 2024-03-22T19:07:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: osdmap e49: 8 total, 8 up, 8 in 2024-03-22T19:07:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:21.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:21 smithi131 ceph-mon[78559]: osdmap e50: 8 total, 8 up, 8 in 2024-03-22T19:07:22.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] boot 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: osdmap e49: 8 total, 8 up, 8 in 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:21 smithi086 ceph-mon[38032]: osdmap e50: 8 total, 8 up, 8 in 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: osd.7 [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] boot 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: osdmap e49: 8 total, 8 up, 8 in 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-22T19:07:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:21 smithi167 ceph-mon[47728]: osdmap e50: 8 total, 8 up, 8 in 2024-03-22T19:07:23.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: osdmap e51: 8 total, 8 up, 8 in 2024-03-22T19:07:23.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:23.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/3660152634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]: dispatch 2024-03-22T19:07:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]: dispatch 2024-03-22T19:07:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]': finished 2024-03-22T19:07:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: osdmap e52: 9 total, 8 up, 9 in 2024-03-22T19:07:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:23.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:23 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/413717994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: osdmap e51: 8 total, 8 up, 8 in 2024-03-22T19:07:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/3660152634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]: dispatch 2024-03-22T19:07:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]: dispatch 2024-03-22T19:07:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]': finished 2024-03-22T19:07:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: osdmap e52: 9 total, 8 up, 9 in 2024-03-22T19:07:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:23.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:23 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/413717994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: osdmap e51: 8 total, 8 up, 8 in 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/3660152634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]: dispatch 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]: dispatch 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3b7bdec-dde1-4853-9969-3de47c4ba84e"}]': finished 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: osdmap e52: 9 total, 8 up, 9 in 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:23 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/413717994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:24.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:24 smithi167 ceph-mon[47728]: osdmap e53: 9 total, 8 up, 9 in 2024-03-22T19:07:24.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:24.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:24 smithi131 ceph-mon[78559]: osdmap e53: 9 total, 8 up, 9 in 2024-03-22T19:07:24.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:24.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:24 smithi086 ceph-mon[38032]: osdmap e53: 9 total, 8 up, 9 in 2024-03-22T19:07:24.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:25 smithi167 ceph-mon[47728]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:25 smithi131 ceph-mon[78559]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:25.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:25 smithi086 ceph-mon[38032]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:27.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:27 smithi167 ceph-mon[47728]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:27 smithi131 ceph-mon[78559]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:27 smithi086 ceph-mon[38032]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:29.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:29 smithi167 ceph-mon[47728]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:29.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-22T19:07:29.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:29.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:29 smithi167 ceph-mon[47728]: Deploying daemon osd.8 on smithi167 2024-03-22T19:07:29.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:29 smithi131 ceph-mon[78559]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:29.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:29 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-22T19:07:29.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:29 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:29.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:29 smithi131 ceph-mon[78559]: Deploying daemon osd.8 on smithi167 2024-03-22T19:07:29.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:29 smithi086 ceph-mon[38032]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:29.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-22T19:07:29.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:29.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:29 smithi086 ceph-mon[38032]: Deploying daemon osd.8 on smithi167 2024-03-22T19:07:31.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:31 smithi131 ceph-mon[78559]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:31.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:31 smithi086 ceph-mon[38032]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:31 smithi167 ceph-mon[47728]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:33.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:33.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:33.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:33.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:32 smithi131 ceph-mon[78559]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:33.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:33.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:33.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:33.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:32 smithi086 ceph-mon[38032]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:33.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:33.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:33.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:33.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:32 smithi167 ceph-mon[47728]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:34.173 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-03-22T19:07:34.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.171+0000 7f22317fa700 1 -- 172.21.15.167:0/3964341911 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2234068e30 con 0x7f221c0a0a90 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f221c0a0a90 msgr2=0x7f221c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f221c0a0a90 0x7f221c0a2f50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2228003790 tx=0x7f2228005fd0 comp rx=0 tx=0).stop 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 msgr2=0x7f22341027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 0x7f22341027f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b970 tx=0x7f222c00bc80 comp rx=0 tx=0).stop 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 shutdown_connections 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f221c0a0a90 0x7f221c0a2f50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234101870 0x7f22341027f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22340ffcd0 0x7f2234103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 --2- 172.21.15.167:0/3964341911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2234100ed0 0x7f22341022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 >> 172.21.15.167:0/3964341911 conn(0x7f22340fb270 msgr2=0x7f2234068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 shutdown_connections 2024-03-22T19:07:34.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:07:34.173+0000 7f223a7e3700 1 -- 172.21.15.167:0/3964341911 wait complete. 2024-03-22T19:07:34.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:34.703 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.8.service 2024-03-22T19:07:34.705 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-03-22T19:07:34.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-22T19:07:34.735 INFO:journalctl@ceph.osd.8.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:07:34.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:34.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:35.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:07:35.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:35 smithi167 ceph-mon[47728]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:35.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:35 smithi131 ceph-mon[78559]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:36.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:35 smithi086 ceph-mon[38032]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:36.328 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:07:35 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8[70823]: 2024-03-22T19:07:35.903+0000 7fc1aade5640 -1 osd.8 0 log_to_monitors true 2024-03-22T19:07:36.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:36 smithi167 ceph-mon[47728]: from='osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-22T19:07:36.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:36 smithi167 ceph-mon[47728]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-22T19:07:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-22T19:07:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-22T19:07:36.882 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-22T19:07:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-22T19:07:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0335199 s, 313 MB/s 2024-03-22T19:07:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-22T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:36 smithi086 ceph-mon[38032]: from='osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-22T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:36 smithi086 ceph-mon[38032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-22T19:07:36.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:36 smithi131 ceph-mon[78559]: from='osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-22T19:07:36.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:36 smithi131 ceph-mon[78559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-22T19:07:37.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-03-22T19:07:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: osdmap e54: 9 total, 8 up, 9 in 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:07:37.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:37.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:37 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: osdmap e54: 9 total, 8 up, 9 in 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:37.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:37.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:37.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:37.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:37.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:37.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:37 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:38.078 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8[70823]: 2024-03-22T19:07:37.634+0000 7fc1a0449700 -1 osd.8 0 waiting for initial osdmap 2024-03-22T19:07:38.078 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8[70823]: 2024-03-22T19:07:37.647+0000 7fc19a23a700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: osdmap e54: 9 total, 8 up, 9 in 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:38.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:38.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:37 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 -- 172.21.15.86:0/680554662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568102980 msgr2=0x7f8568109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 --2- 172.21.15.86:0/680554662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568102980 0x7f8568109230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f856000b0d0 tx=0x7f856000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 -- 172.21.15.86:0/680554662 shutdown_connections 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 --2- 172.21.15.86:0/680554662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568102980 0x7f8568109230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 --2- 172.21.15.86:0/680554662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568100de0 0x7f8568101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 --2- 172.21.15.86:0/680554662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 0x7f8568102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 -- 172.21.15.86:0/680554662 >> 172.21.15.86:0/680554662 conn(0x7f85680fc380 msgr2=0x7f85680fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 -- 172.21.15.86:0/680554662 shutdown_connections 2024-03-22T19:07:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.815+0000 7f856d800700 1 -- 172.21.15.86:0/680554662 wait complete. 2024-03-22T19:07:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 Processor -- start 2024-03-22T19:07:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 -- start start 2024-03-22T19:07:38.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 0x7f8568074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:38.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 0x7f85680731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:38.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568102980 0x7f85680736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:38.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8568073e80 con 0x7f8568100de0 2024-03-22T19:07:38.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8568073ff0 con 0x7f8568101fe0 2024-03-22T19:07:38.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f856d800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85681a0ed0 con 0x7f8568102980 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f8566a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 0x7f8568074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.817+0000 7f8566a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 0x7f8568074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57676/0 (socket says 172.21.15.86:57676) 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 -- 172.21.15.86:0/489937017 learned_addr learned my addr 172.21.15.86:0/489937017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8567400700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568102980 0x7f85680736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566000700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 0x7f85680731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 -- 172.21.15.86:0/489937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568102980 msgr2=0x7f85680736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568102980 0x7f85680736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 -- 172.21.15.86:0/489937017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 msgr2=0x7f85680731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 0x7f85680731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 -- 172.21.15.86:0/489937017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560009d00 con 0x7f8568100de0 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8567400700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568102980 0x7f85680736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566000700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 0x7f85680731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:07:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f8566a00700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 0x7f8568074b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f855800e950 tx=0x7f855800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558009770 con 0x7f8568100de0 2024-03-22T19:07:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85580044f0 con 0x7f8568100de0 2024-03-22T19:07:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558003e00 con 0x7f8568100de0 2024-03-22T19:07:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85681a1160 con 0x7f8568100de0 2024-03-22T19:07:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.818+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85681a1610 con 0x7f8568100de0 2024-03-22T19:07:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.819+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8568108210 con 0x7f8568100de0 2024-03-22T19:07:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.821+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8558003f80 con 0x7f8568100de0 2024-03-22T19:07:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.822+0000 7f855f400700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f854c0a0a90 0x7f854c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.823+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f8558014070 con 0x7f8568100de0 2024-03-22T19:07:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.823+0000 7f8566000700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f854c0a0a90 0x7f854c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.823+0000 7f8566000700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f854c0a0a90 0x7f854c0a2f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8554009cb0 tx=0x7f8554009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:38.827+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8558095dc0 con 0x7f8568100de0 2024-03-22T19:07:38.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi167 2024-03-22T19:07:38.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi167 to 5704M 2024-03-22T19:07:38.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:07:38.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: osdmap e55: 9 total, 8 up, 9 in 2024-03-22T19:07:38.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:38.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:38.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:38 smithi131 ceph-mon[78559]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi167 2024-03-22T19:07:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi167 to 5704M 2024-03-22T19:07:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:07:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: osdmap e55: 9 total, 8 up, 9 in 2024-03-22T19:07:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:38 smithi086 ceph-mon[38032]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:39.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:39.035+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f85680611f0 con 0x7f854c0a0a90 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi167 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi167 to 5704M 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: osdmap e55: 9 total, 8 up, 9 in 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:38 smithi167 ceph-mon[47728]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-03-22T19:07:39.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] boot 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: osdmap e56: 9 total, 9 up, 9 in 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:39 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:39.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:07:39.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:07:39.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] boot 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: osdmap e56: 9 total, 9 up, 9 in 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:39.992 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:39 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:07:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:07:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: osd.8 [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] boot 2024-03-22T19:07:40.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: osdmap e56: 9 total, 9 up, 9 in 2024-03-22T19:07:40.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-22T19:07:40.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:07:40.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:07:40.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:07:40.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:39 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:40.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:40 smithi086 ceph-mon[38032]: osdmap e57: 9 total, 9 up, 9 in 2024-03-22T19:07:40.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:40 smithi086 ceph-mon[38032]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:40.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:40 smithi131 ceph-mon[78559]: osdmap e57: 9 total, 9 up, 9 in 2024-03-22T19:07:40.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:40 smithi131 ceph-mon[78559]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:40 smithi167 ceph-mon[47728]: osdmap e57: 9 total, 9 up, 9 in 2024-03-22T19:07:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:40 smithi167 ceph-mon[47728]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:41.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:41 smithi131 ceph-mon[78559]: osdmap e58: 9 total, 9 up, 9 in 2024-03-22T19:07:41.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:41 smithi131 ceph-mon[78559]: osdmap e59: 9 total, 9 up, 9 in 2024-03-22T19:07:41.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:41 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1823898153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285c5d25-0d34-4b60-a709-e8ba94d47a45"}]: dispatch 2024-03-22T19:07:41.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:41 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1823898153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285c5d25-0d34-4b60-a709-e8ba94d47a45"}]': finished 2024-03-22T19:07:41.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:41 smithi131 ceph-mon[78559]: osdmap e60: 10 total, 9 up, 10 in 2024-03-22T19:07:41.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:41 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:07:42.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:41 smithi086 ceph-mon[38032]: osdmap e58: 9 total, 9 up, 9 in 2024-03-22T19:07:42.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:41 smithi086 ceph-mon[38032]: osdmap e59: 9 total, 9 up, 9 in 2024-03-22T19:07:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:41 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1823898153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285c5d25-0d34-4b60-a709-e8ba94d47a45"}]: dispatch 2024-03-22T19:07:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:41 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1823898153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285c5d25-0d34-4b60-a709-e8ba94d47a45"}]': finished 2024-03-22T19:07:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:41 smithi086 ceph-mon[38032]: osdmap e60: 10 total, 9 up, 10 in 2024-03-22T19:07:42.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:41 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:07:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:41 smithi167 ceph-mon[47728]: osdmap e58: 9 total, 9 up, 9 in 2024-03-22T19:07:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:41 smithi167 ceph-mon[47728]: osdmap e59: 9 total, 9 up, 9 in 2024-03-22T19:07:42.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:41 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1823898153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285c5d25-0d34-4b60-a709-e8ba94d47a45"}]: dispatch 2024-03-22T19:07:42.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:41 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1823898153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285c5d25-0d34-4b60-a709-e8ba94d47a45"}]': finished 2024-03-22T19:07:42.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:41 smithi167 ceph-mon[47728]: osdmap e60: 10 total, 9 up, 10 in 2024-03-22T19:07:42.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:41 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:07:42.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:42 smithi131 ceph-mon[78559]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-03-22T19:07:42.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:42 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/197764239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:43.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:42 smithi086 ceph-mon[38032]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-03-22T19:07:43.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:42 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/197764239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:43.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:42 smithi167 ceph-mon[47728]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-03-22T19:07:43.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:42 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/197764239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:07:45.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:45 smithi131 ceph-mon[78559]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-22T19:07:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:45 smithi086 ceph-mon[38032]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-22T19:07:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:45 smithi167 ceph-mon[47728]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-22T19:07:47.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:47 smithi131 ceph-mon[78559]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T19:07:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:47 smithi086 ceph-mon[38032]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T19:07:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:47 smithi167 ceph-mon[47728]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T19:07:48.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-22T19:07:48.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:48.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-22T19:07:48.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:48.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-22T19:07:48.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:49.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:49 smithi086 ceph-mon[38032]: Deploying daemon osd.9 on smithi086 2024-03-22T19:07:49.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:49 smithi086 ceph-mon[38032]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-22T19:07:49.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:49 smithi131 ceph-mon[78559]: Deploying daemon osd.9 on smithi086 2024-03-22T19:07:49.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:49 smithi131 ceph-mon[78559]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-22T19:07:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:49 smithi167 ceph-mon[47728]: Deploying daemon osd.9 on smithi086 2024-03-22T19:07:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:49 smithi167 ceph-mon[47728]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-22T19:07:51.444 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:51 smithi086 ceph-mon[38032]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-22T19:07:51.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:51 smithi131 ceph-mon[78559]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-22T19:07:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:51 smithi167 ceph-mon[47728]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-22T19:07:53.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:53 smithi131 ceph-mon[78559]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-22T19:07:53.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:53.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:53.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:53 smithi086 ceph-mon[38032]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-22T19:07:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:53.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:53.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:53 smithi167 ceph-mon[47728]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-22T19:07:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:55.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:55 smithi131 ceph-mon[78559]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:55.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:55 smithi086 ceph-mon[38032]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:55 smithi167 ceph-mon[47728]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.040+0000 7f855f400700 1 -- 172.21.15.86:0/489937017 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f85680611f0 con 0x7f854c0a0a90 2024-03-22T19:07:56.041 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f854c0a0a90 msgr2=0x7f854c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f854c0a0a90 0x7f854c0a2f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8554009cb0 tx=0x7f8554009260 comp rx=0 tx=0).stop 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 msgr2=0x7f8568074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 0x7f8568074b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f855800e950 tx=0x7f855800ed10 comp rx=0 tx=0).stop 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 shutdown_connections 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f854c0a0a90 0x7f854c0a2f50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568102980 0x7f85680736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8568100de0 0x7f8568074b00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 --2- 172.21.15.86:0/489937017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8568101fe0 0x7f85680731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 >> 172.21.15.86:0/489937017 conn(0x7f85680fc380 msgr2=0x7f8568105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 shutdown_connections 2024-03-22T19:07:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:07:56.043+0000 7f856d800700 1 -- 172.21.15.86:0/489937017 wait complete. 2024-03-22T19:07:56.487 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.9.service 2024-03-22T19:07:56.492 INFO:tasks.cephadm:Deploying osd.10 on smithi131 with /dev/vg_nvme/lv_1... 2024-03-22T19:07:56.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-22T19:07:56.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:56.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:56.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:56.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:07:56.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:07:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:07:57.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:57 smithi086 ceph-mon[38032]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:57.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:57 smithi086 ceph-mon[38032]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-22T19:07:57.488 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:07:57 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:07:57.174+0000 7fa2abbfe640 -1 osd.9 0 log_to_monitors true 2024-03-22T19:07:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:57 smithi131 ceph-mon[78559]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:57 smithi131 ceph-mon[78559]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-22T19:07:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:57 smithi167 ceph-mon[47728]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:57 smithi167 ceph-mon[47728]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-22T19:07:58.060 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-22T19:07:58.060 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-22T19:07:58.060 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-03-22T19:07:58.060 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-03-22T19:07:58.060 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149409 s, 702 MB/s 2024-03-22T19:07:58.060 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-03-22T19:07:58.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-03-22T19:07:58.700 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:07:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:59 smithi131 ceph-mon[78559]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-22T19:07:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:59 smithi131 ceph-mon[78559]: osdmap e61: 10 total, 9 up, 10 in 2024-03-22T19:07:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:07:59.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:59 smithi131 ceph-mon[78559]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:07:59.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:07:59 smithi131 ceph-mon[78559]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:59.508 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:07:59.118+0000 7fa2a0200700 -1 osd.9 0 waiting for initial osdmap 2024-03-22T19:07:59.508 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:07:59.134+0000 7fa296000700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:07:59.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-mon[38032]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-22T19:07:59.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-mon[38032]: osdmap e61: 10 total, 9 up, 10 in 2024-03-22T19:07:59.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:07:59.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-mon[38032]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:07:59.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:07:59 smithi086 ceph-mon[38032]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:59 smithi167 ceph-mon[47728]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-22T19:07:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:59 smithi167 ceph-mon[47728]: osdmap e61: 10 total, 9 up, 10 in 2024-03-22T19:07:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:07:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:59 smithi167 ceph-mon[47728]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-22T19:07:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:07:59 smithi167 ceph-mon[47728]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:07:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 -- 172.21.15.131:0/2588766243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8102980 msgr2=0x7f8da8109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 --2- 172.21.15.131:0/2588766243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8102980 0x7f8da8109230 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8da000b0d0 tx=0x7f8da000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:07:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 -- 172.21.15.131:0/2588766243 shutdown_connections 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 --2- 172.21.15.131:0/2588766243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8102980 0x7f8da8109230 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 --2- 172.21.15.131:0/2588766243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da8100de0 0x7f8da8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 --2- 172.21.15.131:0/2588766243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da8101fe0 0x7f8da8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 -- 172.21.15.131:0/2588766243 >> 172.21.15.131:0/2588766243 conn(0x7f8da80fc380 msgr2=0x7f8da80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 -- 172.21.15.131:0/2588766243 shutdown_connections 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.646+0000 7f8dad145700 1 -- 172.21.15.131:0/2588766243 wait complete. 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 Processor -- start 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 -- start start 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 0x7f8da8074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da8101fe0 0x7f8da80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da8102980 0x7f8da80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8073e80 con 0x7f8da8102980 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8073ff0 con 0x7f8da8100de0 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8dad145700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81a0ec0 con 0x7f8da8101fe0 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8da6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 0x7f8da8074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8da6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 0x7f8da8074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36398/0 (socket says 172.21.15.131:36398) 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8da6d9d700 1 -- 172.21.15.131:0/1846019238 learned_addr learned my addr 172.21.15.131:0/1846019238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8da659c700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da8101fe0 0x7f8da80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.647+0000 7f8da759e700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da8102980 0x7f8da80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8da6d9d700 1 -- 172.21.15.131:0/1846019238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da8101fe0 msgr2=0x7f8da80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8da6d9d700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da8101fe0 0x7f8da80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8da6d9d700 1 -- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da8102980 msgr2=0x7f8da80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8da6d9d700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da8102980 0x7f8da80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8da6d9d700 1 -- 172.21.15.131:0/1846019238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009d00 con 0x7f8da8100de0 2024-03-22T19:07:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8da6d9d700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 0x7f8da8074b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000b500 tx=0x7f8d9000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90011830 con 0x7f8da8100de0 2024-03-22T19:07:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da81a1150 con 0x7f8da8100de0 2024-03-22T19:07:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d90011e50 con 0x7f8da8100de0 2024-03-22T19:07:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90010dc0 con 0x7f8da8100de0 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.648+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da81a1650 con 0x7f8da8100de0 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.649+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da81081c0 con 0x7f8da8100de0 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.650+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8d90010380 con 0x7f8da8100de0 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.651+0000 7f8d9ffff700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8d940a0a90 0x7f8d940a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.651+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f8d900cb700 con 0x7f8da8100de0 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.651+0000 7f8da659c700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8d940a0a90 0x7f8d940a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:07:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.652+0000 7f8da659c700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8d940a0a90 0x7f8d940a2f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005950 tx=0x7f8d9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:07:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.654+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d90095990 con 0x7f8da8100de0 2024-03-22T19:07:59.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:07:59.855+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8da80611f0 con 0x7f8d940a0a90 2024-03-22T19:08:00.373 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:08:00.373 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: osdmap e62: 10 total, 9 up, 10 in 2024-03-22T19:08:00.373 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:00.373 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:00.373 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi086 2024-03-22T19:08:00.373 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi086 to 4317M 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:08:00.374 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:00.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: osdmap e62: 10 total, 9 up, 10 in 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi086 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi086 to 4317M 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:00.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:00.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:08:00.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:08:00.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-22T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: osdmap e62: 10 total, 9 up, 10 in 2024-03-22T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi086 2024-03-22T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi086 to 4317M 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:08:00.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:01.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:08:01.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:08:01.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:01.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] boot 2024-03-22T19:08:01.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: osdmap e63: 10 total, 10 up, 10 in 2024-03-22T19:08:01.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:01.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:01 smithi131 ceph-mon[78559]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:08:01.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:08:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:08:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] boot 2024-03-22T19:08:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: osdmap e63: 10 total, 10 up, 10 in 2024-03-22T19:08:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:01 smithi086 ceph-mon[38032]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:08:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:08:01.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:08:01.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:01.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: osd.9 [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] boot 2024-03-22T19:08:01.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: osdmap e63: 10 total, 10 up, 10 in 2024-03-22T19:08:01.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-22T19:08:01.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:01 smithi167 ceph-mon[47728]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-03-22T19:08:02.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:02 smithi131 ceph-mon[78559]: osdmap e64: 10 total, 10 up, 10 in 2024-03-22T19:08:02.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:02 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/1600886338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]: dispatch 2024-03-22T19:08:02.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:02 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]: dispatch 2024-03-22T19:08:02.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:02 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]': finished 2024-03-22T19:08:02.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:02 smithi131 ceph-mon[78559]: osdmap e65: 11 total, 10 up, 11 in 2024-03-22T19:08:02.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:02.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:02 smithi086 ceph-mon[38032]: osdmap e64: 10 total, 10 up, 10 in 2024-03-22T19:08:02.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:02 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/1600886338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]: dispatch 2024-03-22T19:08:02.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:02 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]: dispatch 2024-03-22T19:08:02.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:02 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]': finished 2024-03-22T19:08:02.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:02 smithi086 ceph-mon[38032]: osdmap e65: 11 total, 10 up, 11 in 2024-03-22T19:08:02.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:02 smithi167 ceph-mon[47728]: osdmap e64: 10 total, 10 up, 10 in 2024-03-22T19:08:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:02 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/1600886338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]: dispatch 2024-03-22T19:08:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:02 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]: dispatch 2024-03-22T19:08:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:02 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c9eab90-57b4-4b6c-bed1-8f589dc8d971"}]': finished 2024-03-22T19:08:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:02 smithi167 ceph-mon[47728]: osdmap e65: 11 total, 10 up, 11 in 2024-03-22T19:08:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:03.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:03 smithi131 ceph-mon[78559]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:03.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:03 smithi131 ceph-mon[78559]: from='client.? 172.21.15.131:0/2540856673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:08:03.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:03 smithi086 ceph-mon[38032]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:03.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:03 smithi086 ceph-mon[38032]: from='client.? 172.21.15.131:0/2540856673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:08:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:03 smithi167 ceph-mon[47728]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:03 smithi167 ceph-mon[47728]: from='client.? 172.21.15.131:0/2540856673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:08:05.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:05 smithi131 ceph-mon[78559]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:05.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:05 smithi086 ceph-mon[38032]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:05.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:05 smithi167 ceph-mon[47728]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:07.397 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:07 smithi131 ceph-mon[78559]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:07.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:07 smithi086 ceph-mon[38032]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:07 smithi167 ceph-mon[47728]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:08.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:08 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-22T19:08:08.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:08 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:08.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:08 smithi131 ceph-mon[78559]: Deploying daemon osd.10 on smithi131 2024-03-22T19:08:08.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:08 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-22T19:08:08.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:08 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:08.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:08 smithi086 ceph-mon[38032]: Deploying daemon osd.10 on smithi131 2024-03-22T19:08:08.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:08 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-22T19:08:08.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:08 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:08.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:08 smithi167 ceph-mon[47728]: Deploying daemon osd.10 on smithi131 2024-03-22T19:08:09.362 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:09 smithi131 ceph-mon[78559]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:09.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:09 smithi086 ceph-mon[38032]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:09 smithi167 ceph-mon[47728]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:11.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:11 smithi131 ceph-mon[78559]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:11.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:11 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:11.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:11 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:11.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:11 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:11 smithi086 ceph-mon[38032]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:11 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:11 smithi167 ceph-mon[47728]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:11 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:11 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:11 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.160 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 10 on host 'smithi131' 2024-03-22T19:08:13.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.158+0000 7f8d9ffff700 1 -- 172.21.15.131:0/1846019238 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8da80611f0 con 0x7f8d940a0a90 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8d940a0a90 msgr2=0x7f8d940a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8d940a0a90 0x7f8d940a2f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005950 tx=0x7f8d9800b410 comp rx=0 tx=0).stop 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 msgr2=0x7f8da8074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 0x7f8da8074b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000b500 tx=0x7f8d9000b810 comp rx=0 tx=0).stop 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 shutdown_connections 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8d940a0a90 0x7f8d940a2f50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:13.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da8102980 0x7f8da80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:13.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da8100de0 0x7f8da8074b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:13.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.160+0000 7f8dad145700 1 --2- 172.21.15.131:0/1846019238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da8101fe0 0x7f8da80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:13.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.161+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 >> 172.21.15.131:0/1846019238 conn(0x7f8da80fc380 msgr2=0x7f8da8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:13.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.161+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 shutdown_connections 2024-03-22T19:08:13.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-03-22T19:08:13.161+0000 7f8dad145700 1 -- 172.21.15.131:0/1846019238 wait complete. 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.309 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.310 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.310 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:13.536 DEBUG:teuthology.orchestra.run.smithi131:osd.10> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.10.service 2024-03-22T19:08:13.537 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-03-22T19:08:13.538 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-22T19:08:13.568 INFO:journalctl@ceph.osd.10.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:08:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:13.770 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:08:15.075 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-22T19:08:15.075 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-22T19:08:15.075 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-03-22T19:08:15.075 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-03-22T19:08:15.075 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113244 s, 926 MB/s 2024-03-22T19:08:15.075 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-03-22T19:08:15.344 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:08:15 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:08:15.089+0000 7f0ecefa5640 -1 osd.10 0 log_to_monitors true 2024-03-22T19:08:15.345 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:15 smithi131 ceph-mon[78559]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:15.345 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:15 smithi131 ceph-mon[78559]: from='osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-22T19:08:15.345 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:15 smithi131 ceph-mon[78559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-22T19:08:15.403 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:15 smithi167 ceph-mon[47728]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:15.403 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:15 smithi167 ceph-mon[47728]: from='osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-22T19:08:15.403 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:15 smithi167 ceph-mon[47728]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-22T19:08:15.472 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-03-22T19:08:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:15 smithi086 ceph-mon[38032]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:15 smithi086 ceph-mon[38032]: from='osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-22T19:08:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:15 smithi086 ceph-mon[38032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-22T19:08:15.733 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:08:16.471 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:08:16.213+0000 7f0ec4609700 -1 osd.10 0 waiting for initial osdmap 2024-03-22T19:08:16.471 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:08:16.226+0000 7f0ebebfb700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: osdmap e66: 11 total, 10 up, 11 in 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:16.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:16.473 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:16 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-22T19:08:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: osdmap e66: 11 total, 10 up, 11 in 2024-03-22T19:08:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:08:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:16 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: osdmap e66: 11 total, 10 up, 11 in 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:16.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:16.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:16 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 -- 172.21.15.167:0/3170227799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 msgr2=0x7f61880ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 --2- 172.21.15.167:0/3170227799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f61880ff290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6178009990 tx=0x7f6178009ca0 comp rx=0 tx=0).stop 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 -- 172.21.15.167:0/3170227799 shutdown_connections 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 --2- 172.21.15.167:0/3170227799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61880ff7d0 0x7f61881093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 --2- 172.21.15.167:0/3170227799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61881023b0 0x7f61881027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 --2- 172.21.15.167:0/3170227799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f61880ff290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:16.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 -- 172.21.15.167:0/3170227799 >> 172.21.15.167:0/3170227799 conn(0x7f61880fa9f0 msgr2=0x7f61880fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:16.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 -- 172.21.15.167:0/3170227799 shutdown_connections 2024-03-22T19:08:16.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.695+0000 7f618e378700 1 -- 172.21.15.167:0/3170227799 wait complete. 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 Processor -- start 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 -- start start 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f6188196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61880ff7d0 0x7f61881972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61881023b0 0x7f618819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819e890 con 0x7f61880ff7d0 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819e9d0 con 0x7f61881023b0 2024-03-22T19:08:16.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618e378700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819eb90 con 0x7f61880fee30 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f6188196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f6188196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56238/0 (socket says 172.21.15.167:56238) 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 -- 172.21.15.167:0/441971005 learned_addr learned my addr 172.21.15.167:0/441971005 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f61877fe700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61880ff7d0 0x7f61881972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f618c915700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61881023b0 0x7f618819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 -- 172.21.15.167:0/441971005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61881023b0 msgr2=0x7f618819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61881023b0 0x7f618819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 -- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61880ff7d0 msgr2=0x7f61881972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61880ff7d0 0x7f61881972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:16.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.696+0000 7f6187fff700 1 -- 172.21.15.167:0/441971005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780095f0 con 0x7f61880fee30 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.697+0000 7f6187fff700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f6188196d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f617c00d980 tx=0x7f617c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.697+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c009b40 con 0x7f61880fee30 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.697+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617c00de60 con 0x7f61880fee30 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.697+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618819ee70 con 0x7f61880fee30 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.697+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00f620 con 0x7f61880fee30 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.697+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618819f230 con 0x7f61880fee30 2024-03-22T19:08:16.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.698+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6188108380 con 0x7f61880fee30 2024-03-22T19:08:16.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.699+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f617c010460 con 0x7f61880fee30 2024-03-22T19:08:16.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.700+0000 7f61857fa700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f61700a0a90 0x7f61700a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:16.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.700+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f617c0cb4f0 con 0x7f61880fee30 2024-03-22T19:08:16.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.700+0000 7f61877fe700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f61700a0a90 0x7f61700a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:16.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.701+0000 7f61877fe700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f61700a0a90 0x7f61700a2f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f617800b5c0 tx=0x7f6178005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:16.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.704+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f617c095590 con 0x7f61880fee30 2024-03-22T19:08:16.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:16.904+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f61880611f0 con 0x7f61700a0a90 2024-03-22T19:08:17.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi131 2024-03-22T19:08:17.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi131 to 4278M 2024-03-22T19:08:17.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:17.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:08:17.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: osdmap e67: 11 total, 10 up, 11 in 2024-03-22T19:08:17.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] boot 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: osdmap e68: 11 total, 11 up, 11 in 2024-03-22T19:08:17.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi131 2024-03-22T19:08:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi131 to 4278M 2024-03-22T19:08:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:08:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: osdmap e67: 11 total, 10 up, 11 in 2024-03-22T19:08:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] boot 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: osdmap e68: 11 total, 11 up, 11 in 2024-03-22T19:08:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi131 2024-03-22T19:08:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi131 to 4278M 2024-03-22T19:08:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-03-22T19:08:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-03-22T19:08:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: osdmap e67: 11 total, 10 up, 11 in 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: osd.10 [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] boot 2024-03-22T19:08:17.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: osdmap e68: 11 total, 11 up, 11 in 2024-03-22T19:08:17.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-22T19:08:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:18 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:08:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:18 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:08:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:18 smithi131 ceph-mon[78559]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:18 smithi131 ceph-mon[78559]: osdmap e69: 11 total, 11 up, 11 in 2024-03-22T19:08:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:18 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:08:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:18 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:08:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:18 smithi086 ceph-mon[38032]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:18 smithi086 ceph-mon[38032]: osdmap e69: 11 total, 11 up, 11 in 2024-03-22T19:08:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:18 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:08:18.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:18 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:08:18.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:18 smithi167 ceph-mon[47728]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:18.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:18 smithi167 ceph-mon[47728]: osdmap e69: 11 total, 11 up, 11 in 2024-03-22T19:08:19.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:19 smithi086 ceph-mon[38032]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:19 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/2448910294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]: dispatch 2024-03-22T19:08:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:19 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]: dispatch 2024-03-22T19:08:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:19 smithi086 ceph-mon[38032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]': finished 2024-03-22T19:08:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:19 smithi086 ceph-mon[38032]: osdmap e70: 12 total, 11 up, 12 in 2024-03-22T19:08:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:19 smithi167 ceph-mon[47728]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:19 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/2448910294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]: dispatch 2024-03-22T19:08:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:19 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]: dispatch 2024-03-22T19:08:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:19 smithi167 ceph-mon[47728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]': finished 2024-03-22T19:08:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:19 smithi167 ceph-mon[47728]: osdmap e70: 12 total, 11 up, 12 in 2024-03-22T19:08:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:19.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:19 smithi131 ceph-mon[78559]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:19.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:19 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/2448910294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]: dispatch 2024-03-22T19:08:19.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:19 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]: dispatch 2024-03-22T19:08:19.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:19 smithi131 ceph-mon[78559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1101cd32-ff32-47ff-8023-01e657e68b67"}]': finished 2024-03-22T19:08:19.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:19 smithi131 ceph-mon[78559]: osdmap e70: 12 total, 11 up, 12 in 2024-03-22T19:08:19.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:20.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:20 smithi086 ceph-mon[38032]: from='client.? 172.21.15.167:0/2368565060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:08:20.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:20 smithi086 ceph-mon[38032]: osdmap e71: 12 total, 11 up, 12 in 2024-03-22T19:08:20.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:20 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:20 smithi167 ceph-mon[47728]: from='client.? 172.21.15.167:0/2368565060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:08:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:20 smithi167 ceph-mon[47728]: osdmap e71: 12 total, 11 up, 12 in 2024-03-22T19:08:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:20 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:20.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:20 smithi131 ceph-mon[78559]: from='client.? 172.21.15.167:0/2368565060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T19:08:20.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:20 smithi131 ceph-mon[78559]: osdmap e71: 12 total, 11 up, 12 in 2024-03-22T19:08:20.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:20 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:21.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:21 smithi086 ceph-mon[38032]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:21 smithi086 ceph-mon[38032]: osdmap e72: 12 total, 11 up, 12 in 2024-03-22T19:08:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:21 smithi167 ceph-mon[47728]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:21 smithi167 ceph-mon[47728]: osdmap e72: 12 total, 11 up, 12 in 2024-03-22T19:08:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:21 smithi131 ceph-mon[78559]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:21 smithi131 ceph-mon[78559]: osdmap e72: 12 total, 11 up, 12 in 2024-03-22T19:08:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:23 smithi086 ceph-mon[38032]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T19:08:23.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:23 smithi167 ceph-mon[47728]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T19:08:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:23 smithi131 ceph-mon[78559]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T19:08:25.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:25 smithi167 ceph-mon[47728]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-22T19:08:25.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:25 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-22T19:08:25.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:25 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:25 smithi086 ceph-mon[38032]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-22T19:08:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:25 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-22T19:08:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:25 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:25 smithi131 ceph-mon[78559]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-22T19:08:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:25 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-22T19:08:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:25 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:26.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:26 smithi167 ceph-mon[47728]: Deploying daemon osd.11 on smithi167 2024-03-22T19:08:26.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:26 smithi086 ceph-mon[38032]: Deploying daemon osd.11 on smithi167 2024-03-22T19:08:26.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:26 smithi131 ceph-mon[78559]: Deploying daemon osd.11 on smithi167 2024-03-22T19:08:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:27 smithi086 ceph-mon[38032]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-22T19:08:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:27 smithi167 ceph-mon[47728]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-22T19:08:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:27 smithi131 ceph-mon[78559]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-22T19:08:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:29 smithi131 ceph-mon[78559]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-22T19:08:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:29 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:29 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:29 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:29.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:29 smithi086 ceph-mon[38032]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-22T19:08:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:29 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:29 smithi167 ceph-mon[47728]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-22T19:08:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:29 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:30.811 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-03-22T19:08:30.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.809+0000 7f61857fa700 1 -- 172.21.15.167:0/441971005 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f61880611f0 con 0x7f61700a0a90 2024-03-22T19:08:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.811+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f61700a0a90 msgr2=0x7f61700a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.811+0000 7f618e378700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f61700a0a90 0x7f61700a2f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f617800b5c0 tx=0x7f6178005fb0 comp rx=0 tx=0).stop 2024-03-22T19:08:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.811+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 msgr2=0x7f6188196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.811+0000 7f618e378700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f6188196d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f617c00d980 tx=0x7f617c00dd40 comp rx=0 tx=0).stop 2024-03-22T19:08:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 shutdown_connections 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f61700a0a90 0x7f61700a2f50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61881023b0 0x7f618819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61880fee30 0x7f6188196d90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 --2- 172.21.15.167:0/441971005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61880ff7d0 0x7f61881972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 >> 172.21.15.167:0/441971005 conn(0x7f61880fa9f0 msgr2=0x7f61881054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 shutdown_connections 2024-03-22T19:08:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:30.812+0000 7f618e378700 1 -- 172.21.15.167:0/441971005 wait complete. 2024-03-22T19:08:30.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:30 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:30.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:30 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:30.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:30 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:30.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:30 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:30.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:30 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:30.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:30 smithi167 ceph-mon[47728]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-22T19:08:31.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:30 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:31.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:30 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:31.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:30 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:31.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:30 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:31.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:30 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:31.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:30 smithi131 ceph-mon[78559]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-22T19:08:31.224 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.11.service 2024-03-22T19:08:31.226 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-22T19:08:31.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T19:08:31.258 INFO:journalctl@ceph.osd.11.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:08:31.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:30 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:31.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:30 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:31.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:30 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:31.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:30 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:31.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:30 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:31.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:30 smithi086 ceph-mon[38032]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-22T19:08:31.591 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:08:32.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:31 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:32.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:31 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:32.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:31 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:32.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:31 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:32.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:31 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:32.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:31 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:32.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:31 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:32.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:31 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:32.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:31 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:32.785 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:08:32 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:08:32.393+0000 7f826d4da640 -1 osd.11 0 log_to_monitors true 2024-03-22T19:08:32.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 -- 172.21.15.86:0/3254994765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 msgr2=0x7fa494101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 --2- 172.21.15.86:0/3254994765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494101200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa4840099c0 tx=0x7fa484009cd0 comp rx=0 tx=0).stop 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 -- 172.21.15.86:0/3254994765 shutdown_connections 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 --2- 172.21.15.86:0/3254994765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494102980 0x7fa494109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 --2- 172.21.15.86:0/3254994765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494101200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 --2- 172.21.15.86:0/3254994765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494101fe0 0x7fa494102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.870+0000 7fa49a000700 1 -- 172.21.15.86:0/3254994765 >> 172.21.15.86:0/3254994765 conn(0x7fa4940fc380 msgr2=0x7fa4940fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.871+0000 7fa49a000700 1 -- 172.21.15.86:0/3254994765 shutdown_connections 2024-03-22T19:08:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.871+0000 7fa49a000700 1 -- 172.21.15.86:0/3254994765 wait complete. 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 Processor -- start 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 -- start start 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494101fe0 0x7fa4941972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494102980 0x7fa49419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49419e850 con 0x7fa494100de0 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49419e990 con 0x7fa494102980 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa49a000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49419eb50 con 0x7fa494101fe0 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa493400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa493400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43044/0 (socket says 172.21.15.86:43044) 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa493400700 1 -- 172.21.15.86:0/744996690 learned_addr learned my addr 172.21.15.86:0/744996690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.872+0000 7fa492a00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494101fe0 0x7fa4941972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493e00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494102980 0x7fa49419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493400700 1 -- 172.21.15.86:0/744996690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494101fe0 msgr2=0x7fa4941972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493400700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494101fe0 0x7fa4941972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493400700 1 -- 172.21.15.86:0/744996690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494102980 msgr2=0x7fa49419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493400700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494102980 0x7fa49419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493400700 1 -- 172.21.15.86:0/744996690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840095f0 con 0x7fa494100de0 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa492a00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494101fe0 0x7fa4941972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493e00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494102980 0x7fa49419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa493400700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494196da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa484003730 tx=0x7fa48400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48401d070 con 0x7fa494100de0 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa484005050 con 0x7fa494100de0 2024-03-22T19:08:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49419edd0 con 0x7fa494100de0 2024-03-22T19:08:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.873+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484004af0 con 0x7fa494100de0 2024-03-22T19:08:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.874+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49419f190 con 0x7fa494100de0 2024-03-22T19:08:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.874+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa494108260 con 0x7fa494100de0 2024-03-22T19:08:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.876+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa4840051c0 con 0x7fa494100de0 2024-03-22T19:08:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.877+0000 7fa48be00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4780a0b10 0x7fa4780a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.878+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fa4840ccbe0 con 0x7fa494100de0 2024-03-22T19:08:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.878+0000 7fa492a00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4780a0b10 0x7fa4780a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.878+0000 7fa492a00700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4780a0b10 0x7fa4780a2fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa480011490 tx=0x7fa480009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:32.880+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa484096b70 con 0x7fa494100de0 2024-03-22T19:08:33.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:32 smithi086 ceph-mon[38032]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-22T19:08:33.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:32 smithi086 ceph-mon[38032]: from='osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-22T19:08:33.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:32 smithi086 ceph-mon[38032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-22T19:08:33.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:32 smithi167 ceph-mon[47728]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-22T19:08:33.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:32 smithi167 ceph-mon[47728]: from='osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-22T19:08:33.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:32 smithi167 ceph-mon[47728]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-22T19:08:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.083+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa49404ea80 con 0x7fa494100de0 2024-03-22T19:08:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.083+0000 7fa48be00700 1 -- 172.21.15.86:0/744996690 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa48409a190 con 0x7fa494100de0 2024-03-22T19:08:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:33.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4780a0b10 msgr2=0x7fa4780a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:33.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4780a0b10 0x7fa4780a2fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa480011490 tx=0x7fa480009260 comp rx=0 tx=0).stop 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 msgr2=0x7fa494196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494196da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa484003730 tx=0x7fa48400f690 comp rx=0 tx=0).stop 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 shutdown_connections 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4780a0b10 0x7fa4780a2fd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa494102980 0x7fa49419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494100de0 0x7fa494196da0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 --2- 172.21.15.86:0/744996690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa494101fe0 0x7fa4941972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 >> 172.21.15.86:0/744996690 conn(0x7fa4940fc380 msgr2=0x7fa4941053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 shutdown_connections 2024-03-22T19:08:33.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:33.086+0000 7fa49a000700 1 -- 172.21.15.86:0/744996690 wait complete. 2024-03-22T19:08:33.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:32 smithi131 ceph-mon[78559]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-22T19:08:33.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:32 smithi131 ceph-mon[78559]: from='osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-22T19:08:33.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:32 smithi131 ceph-mon[78559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-22T19:08:33.535 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1711134497,"num_in_osds":12,"osd_in_since":1711134498,"num_remapped_pgs":0} 2024-03-22T19:08:34.078 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:08:33.799+0000 7f8262b3e700 -1 osd.11 0 waiting for initial osdmap 2024-03-22T19:08:34.078 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:08:33.815+0000 7f825d931700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T19:08:34.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-22T19:08:34.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: osdmap e73: 12 total, 11 up, 12 in 2024-03-22T19:08:34.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/744996690' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: Detected new or changed devices on smithi167 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi167 to 4278M 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:34.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:34.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:33 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: osdmap e73: 12 total, 11 up, 12 in 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/744996690' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: Detected new or changed devices on smithi167 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi167 to 4278M 2024-03-22T19:08:34.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:34.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:34.223 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:33 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-22T19:08:34.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: osdmap e73: 12 total, 11 up, 12 in 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/744996690' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: Detected new or changed devices on smithi167 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi167 to 4278M 2024-03-22T19:08:34.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:34.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:34.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:33 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:34.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T19:08:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:08:35.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:34 smithi086 ceph-mon[38032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:08:35.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:34 smithi086 ceph-mon[38032]: osdmap e74: 12 total, 11 up, 12 in 2024-03-22T19:08:35.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:35.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:34 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:35.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:34 smithi086 ceph-mon[38032]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:35.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:34 smithi167 ceph-mon[47728]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:08:35.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:34 smithi167 ceph-mon[47728]: osdmap e74: 12 total, 11 up, 12 in 2024-03-22T19:08:35.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:35.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:34 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:35.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:34 smithi167 ceph-mon[47728]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:35.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:34 smithi131 ceph-mon[78559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-03-22T19:08:35.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:34 smithi131 ceph-mon[78559]: osdmap e74: 12 total, 11 up, 12 in 2024-03-22T19:08:35.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:35.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:34 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:35.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:34 smithi131 ceph-mon[78559]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-03-22T19:08:36.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:35 smithi167 ceph-mon[47728]: purged_snaps scrub starts 2024-03-22T19:08:36.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:35 smithi167 ceph-mon[47728]: purged_snaps scrub ok 2024-03-22T19:08:36.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:35 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:36.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:35 smithi167 ceph-mon[47728]: osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] boot 2024-03-22T19:08:36.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:35 smithi167 ceph-mon[47728]: osdmap e75: 12 total, 12 up, 12 in 2024-03-22T19:08:36.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:35 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 -- 172.21.15.86:0/1796385001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 msgr2=0x7fb374102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 --2- 172.21.15.86:0/1796385001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 0x7fb374102440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb3600098f0 tx=0x7fb360009c00 comp rx=0 tx=0).stop 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 -- 172.21.15.86:0/1796385001 shutdown_connections 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 --2- 172.21.15.86:0/1796385001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374102980 0x7fb374109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 --2- 172.21.15.86:0/1796385001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374100de0 0x7fb374101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 --2- 172.21.15.86:0/1796385001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 0x7fb374102440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 -- 172.21.15.86:0/1796385001 >> 172.21.15.86:0/1796385001 conn(0x7fb3740fc380 msgr2=0x7fb3740fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 -- 172.21.15.86:0/1796385001 shutdown_connections 2024-03-22T19:08:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.106+0000 7fb37b000700 1 -- 172.21.15.86:0/1796385001 wait complete. 2024-03-22T19:08:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.107+0000 7fb37b000700 1 Processor -- start 2024-03-22T19:08:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.107+0000 7fb37b000700 1 -- start start 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.107+0000 7fb37b000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 0x7fb374196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb37b000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 0x7fb374197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb37b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374102980 0x7fb37419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb37b000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37419e8c0 con 0x7fb374100de0 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb37b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37419ea00 con 0x7fb374102980 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb37b000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37419eb90 con 0x7fb374101fe0 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb373e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 0x7fb374196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb373e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 0x7fb374196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43072/0 (socket says 172.21.15.86:43072) 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb373e00700 1 -- 172.21.15.86:0/1672158776 learned_addr learned my addr 172.21.15.86:0/1672158776 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb379000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374102980 0x7fb37419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb373400700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 0x7fb374197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb379000700 1 -- 172.21.15.86:0/1672158776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 msgr2=0x7fb374197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb379000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 0x7fb374197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb379000700 1 -- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 msgr2=0x7fb374196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb379000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 0x7fb374196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb379000700 1 -- 172.21.15.86:0/1672158776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c009c40 con 0x7fb374102980 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.108+0000 7fb373e00700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 0x7fb374196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.109+0000 7fb379000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374102980 0x7fb37419e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00efb0 tx=0x7fb36c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.109+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c003a00 con 0x7fb374102980 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.109+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb36c00f040 con 0x7fb374102980 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.109+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c004590 con 0x7fb374102980 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.109+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb360009520 con 0x7fb374102980 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.109+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37419f190 con 0x7fb374102980 2024-03-22T19:08:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.110+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3741083b0 con 0x7fb374102980 2024-03-22T19:08:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.112+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fb36c0046f0 con 0x7fb374102980 2024-03-22T19:08:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.113+0000 7fb370c00700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3580a0a90 0x7fb3580a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.113+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb36c0cbb60 con 0x7fb374102980 2024-03-22T19:08:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.114+0000 7fb373e00700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3580a0a90 0x7fb3580a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.114+0000 7fb373e00700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3580a0a90 0x7fb3580a2f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb364009fd0 tx=0x7fb364009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.116+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb36c0959c0 con 0x7fb374102980 2024-03-22T19:08:36.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:35 smithi131 ceph-mon[78559]: purged_snaps scrub starts 2024-03-22T19:08:36.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:35 smithi131 ceph-mon[78559]: purged_snaps scrub ok 2024-03-22T19:08:36.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:35 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:36.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:35 smithi131 ceph-mon[78559]: osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] boot 2024-03-22T19:08:36.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:35 smithi131 ceph-mon[78559]: osdmap e75: 12 total, 12 up, 12 in 2024-03-22T19:08:36.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:35 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:36.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:35 smithi086 ceph-mon[38032]: purged_snaps scrub starts 2024-03-22T19:08:36.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:35 smithi086 ceph-mon[38032]: purged_snaps scrub ok 2024-03-22T19:08:36.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:35 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:36.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:35 smithi086 ceph-mon[38032]: osd.11 [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] boot 2024-03-22T19:08:36.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:35 smithi086 ceph-mon[38032]: osdmap e75: 12 total, 12 up, 12 in 2024-03-22T19:08:36.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:35 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-22T19:08:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.320+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb37404ea80 con 0x7fb374102980 2024-03-22T19:08:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.321+0000 7fb370c00700 1 -- 172.21.15.86:0/1672158776 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fb36c098fe0 con 0x7fb374102980 2024-03-22T19:08:36.323 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:36.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3580a0a90 msgr2=0x7fb3580a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:36.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3580a0a90 0x7fb3580a2f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb364009fd0 tx=0x7fb364009310 comp rx=0 tx=0).stop 2024-03-22T19:08:36.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374102980 msgr2=0x7fb37419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374102980 0x7fb37419e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00efb0 tx=0x7fb36c00c3a0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 shutdown_connections 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3580a0a90 0x7fb3580a2f50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb374102980 0x7fb37419e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb374100de0 0x7fb374196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 --2- 172.21.15.86:0/1672158776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374101fe0 0x7fb374197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.323+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 >> 172.21.15.86:0/1672158776 conn(0x7fb3740fc380 msgr2=0x7fb3741054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.324+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 shutdown_connections 2024-03-22T19:08:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:36.324+0000 7fb37b000700 1 -- 172.21.15.86:0/1672158776 wait complete. 2024-03-22T19:08:36.766 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1711134514,"num_in_osds":12,"osd_in_since":1711134498,"num_remapped_pgs":0} 2024-03-22T19:08:36.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T19:08:37.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:36 smithi086 ceph-mon[38032]: osdmap e76: 12 total, 12 up, 12 in 2024-03-22T19:08:37.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:36 smithi086 ceph-mon[38032]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:37.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:36 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1672158776' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T19:08:37.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:36 smithi167 ceph-mon[47728]: osdmap e76: 12 total, 12 up, 12 in 2024-03-22T19:08:37.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:36 smithi167 ceph-mon[47728]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:37.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:36 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1672158776' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T19:08:37.120 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:08:37.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:36 smithi131 ceph-mon[78559]: osdmap e76: 12 total, 12 up, 12 in 2024-03-22T19:08:37.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:36 smithi131 ceph-mon[78559]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:37.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:36 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1672158776' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T19:08:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 -- 172.21.15.86:0/2820853479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101790 msgr2=0x7fb3241057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 --2- 172.21.15.86:0/2820853479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101790 0x7fb3241057e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310009ca0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 -- 172.21.15.86:0/2820853479 shutdown_connections 2024-03-22T19:08:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 --2- 172.21.15.86:0/2820853479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324105ec0 0x7fb32410da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 --2- 172.21.15.86:0/2820853479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb324100de0 0x7fb3241011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 --2- 172.21.15.86:0/2820853479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101790 0x7fb3241057e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 -- 172.21.15.86:0/2820853479 >> 172.21.15.86:0/2820853479 conn(0x7fb3240fc6b0 msgr2=0x7fb3240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.298+0000 7fb32a600700 1 -- 172.21.15.86:0/2820853479 shutdown_connections 2024-03-22T19:08:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 -- 172.21.15.86:0/2820853479 wait complete. 2024-03-22T19:08:38.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 Processor -- start 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 -- start start 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb324100de0 0x7fb324197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 0x7fb3241975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324105ec0 0x7fb32419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32419bf70 con 0x7fb324105ec0 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32419c0e0 con 0x7fb324100de0 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb32a600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32419c2a0 con 0x7fb324101790 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb323e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 0x7fb3241975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb329c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324105ec0 0x7fb32419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb323e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 0x7fb3241975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42630/0 (socket says 172.21.15.86:42630) 2024-03-22T19:08:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb323e00700 1 -- 172.21.15.86:0/662975833 learned_addr learned my addr 172.21.15.86:0/662975833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.300+0000 7fb329200700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb324100de0 0x7fb324197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb323e00700 1 -- 172.21.15.86:0/662975833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb324100de0 msgr2=0x7fb324197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb323e00700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb324100de0 0x7fb324197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb323e00700 1 -- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324105ec0 msgr2=0x7fb32419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb323e00700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324105ec0 0x7fb32419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb323e00700 1 -- 172.21.15.86:0/662975833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100095f0 con 0x7fb324101790 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb329c00700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324105ec0 0x7fb32419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb323e00700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 0x7fb3241975b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb310009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31001d070 con 0x7fb324101790 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb310021410 con 0x7fb324101790 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310017650 con 0x7fb324101790 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32419c520 con 0x7fb324101790 2024-03-22T19:08:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.301+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3241a3c10 con 0x7fb324101790 2024-03-22T19:08:38.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.302+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3241091e0 con 0x7fb324101790 2024-03-22T19:08:38.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.304+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fb310021580 con 0x7fb324101790 2024-03-22T19:08:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.305+0000 7fb321600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb30c0a0a90 0x7fb30c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.306+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3100cce70 con 0x7fb324101790 2024-03-22T19:08:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.306+0000 7fb329200700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb30c0a0a90 0x7fb30c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.306+0000 7fb329200700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb30c0a0a90 0x7fb30c0a2f50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb318005950 tx=0x7fb31800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:38.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.310+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb310096cd0 con 0x7fb324101790 2024-03-22T19:08:38.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.515+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3241022c0 con 0x7fb324101790 2024-03-22T19:08:38.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.516+0000 7fb321600700 1 -- 172.21.15.86:0/662975833 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18422 (secure 0 0 0) 0x7fb31009a2f0 con 0x7fb324101790 2024-03-22T19:08:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:38.519 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:08:35.803767+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:08:38.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb30c0a0a90 msgr2=0x7fb30c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:38.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb30c0a0a90 0x7fb30c0a2f50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb318005950 tx=0x7fb31800b410 comp rx=0 tx=0).stop 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 msgr2=0x7fb3241975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 0x7fb3241975b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb310009580 comp rx=0 tx=0).stop 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 shutdown_connections 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb30c0a0a90 0x7fb30c0a2f50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324105ec0 0x7fb32419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb324100de0 0x7fb324197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 --2- 172.21.15.86:0/662975833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb324101790 0x7fb3241975b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 >> 172.21.15.86:0/662975833 conn(0x7fb3240fc6b0 msgr2=0x7fb3240fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 shutdown_connections 2024-03-22T19:08:38.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:38.519+0000 7fb32a600700 1 -- 172.21.15.86:0/662975833 wait complete. 2024-03-22T19:08:38.949 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:08:38.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-22T19:08:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:08:39.450 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:39 smithi086 ceph-mon[38032]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:39.450 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:39 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/662975833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:08:39.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:39 smithi131 ceph-mon[78559]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:39.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:39 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/662975833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:08:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:39 smithi167 ceph-mon[47728]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:39 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/662975833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:08:40.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 -- 172.21.15.86:0/835338700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638100fb0 msgr2=0x7fe638101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:40.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 --2- 172.21.15.86:0/835338700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638100fb0 0x7fe638101390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe628009990 tx=0x7fe628009ca0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 -- 172.21.15.86:0/835338700 shutdown_connections 2024-03-22T19:08:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 --2- 172.21.15.86:0/835338700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638106090 0x7fe63810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 --2- 172.21.15.86:0/835338700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638100fb0 0x7fe638101390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 --2- 172.21.15.86:0/835338700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe638101960 0x7fe6381059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 -- 172.21.15.86:0/835338700 >> 172.21.15.86:0/835338700 conn(0x7fe6380fc800 msgr2=0x7fe6380fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 -- 172.21.15.86:0/835338700 shutdown_connections 2024-03-22T19:08:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.474+0000 7fe63cc00700 1 -- 172.21.15.86:0/835338700 wait complete. 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.475+0000 7fe63cc00700 1 Processor -- start 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.475+0000 7fe63cc00700 1 -- start start 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe63cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100fb0 0x7fe638199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe63cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 0x7fe638199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe63cc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe638106090 0x7fe63819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe63cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63819e320 con 0x7fe638101960 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe63cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63819e490 con 0x7fe638100fb0 2024-03-22T19:08:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe63cc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63819e650 con 0x7fe638106090 2024-03-22T19:08:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 0x7fe638199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 0x7fe638199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60662/0 (socket says 172.21.15.86:60662) 2024-03-22T19:08:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100fb0 0x7fe638199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 -- 172.21.15.86:0/95068154 learned_addr learned my addr 172.21.15.86:0/95068154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe636800700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe638106090 0x7fe63819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 -- 172.21.15.86:0/95068154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe638106090 msgr2=0x7fe63819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe638106090 0x7fe63819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 -- 172.21.15.86:0/95068154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100fb0 msgr2=0x7fe638199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100fb0 0x7fe638199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635400700 1 -- 172.21.15.86:0/95068154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280095f0 con 0x7fe638101960 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.476+0000 7fe635e00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100fb0 0x7fe638199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.477+0000 7fe635400700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 0x7fe638199960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe62400e970 tx=0x7fe62400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.477+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624009790 con 0x7fe638101960 2024-03-22T19:08:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.477+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe62400c7e0 con 0x7fe638101960 2024-03-22T19:08:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.477+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6240054e0 con 0x7fe638101960 2024-03-22T19:08:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.477+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63819e930 con 0x7fe638101960 2024-03-22T19:08:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.477+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6381a6190 con 0x7fe638101960 2024-03-22T19:08:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.478+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe638102490 con 0x7fe638101960 2024-03-22T19:08:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.480+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe62400c950 con 0x7fe638101960 2024-03-22T19:08:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.482+0000 7fe62ea00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe61c0a0b10 0x7fe61c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.482+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe624014070 con 0x7fe638101960 2024-03-22T19:08:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.482+0000 7fe635e00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe61c0a0b10 0x7fe61c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.483+0000 7fe635e00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe61c0a0b10 0x7fe61c0a2fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe628006010 tx=0x7fe62800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.486+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe624095c10 con 0x7fe638101960 2024-03-22T19:08:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.686+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe63804ea80 con 0x7fe638101960 2024-03-22T19:08:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.686+0000 7fe62ea00700 1 -- 172.21.15.86:0/95068154 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe624099230 con 0x7fe638101960 2024-03-22T19:08:40.687 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:08:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.688+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe61c0a0b10 msgr2=0x7fe61c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.688+0000 7fe63cc00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe61c0a0b10 0x7fe61c0a2fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe628006010 tx=0x7fe62800b260 comp rx=0 tx=0).stop 2024-03-22T19:08:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.688+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 msgr2=0x7fe638199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.688+0000 7fe63cc00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 0x7fe638199960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe62400e970 tx=0x7fe62400ec80 comp rx=0 tx=0).stop 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 shutdown_connections 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe61c0a0b10 0x7fe61c0a2fd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe638106090 0x7fe63819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100fb0 0x7fe638199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 --2- 172.21.15.86:0/95068154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe638101960 0x7fe638199960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 >> 172.21.15.86:0/95068154 conn(0x7fe6380fc800 msgr2=0x7fe6380fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 shutdown_connections 2024-03-22T19:08:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:40.689+0000 7fe63cc00700 1 -- 172.21.15.86:0/95068154 wait complete. 2024-03-22T19:08:41.098 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.b on smithi131 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.e on smithi131 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.h on smithi131 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.k on smithi131 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-03-22T19:08:41.099 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-03-22T19:08:41.100 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-03-22T19:08:41.333 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:08:41.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:41 smithi167 ceph-mon[47728]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:41.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:41 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/95068154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:08:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:41 smithi131 ceph-mon[78559]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:41.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:41 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/95068154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:08:41.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:41 smithi086 ceph-mon[38032]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:41.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:41 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/95068154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 -- 172.21.15.167:0/1997358244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4100ef0 msgr2=0x7f9bf4101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/1997358244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4100ef0 0x7f9bf4101350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009990 tx=0x7f9bf0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 -- 172.21.15.167:0/1997358244 shutdown_connections 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/1997358244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4101890 0x7f9bf4108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/1997358244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf40ffcf0 0x7f9bf4100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/1997358244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4100ef0 0x7f9bf4101350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.266+0000 7f9bfbe30700 1 -- 172.21.15.167:0/1997358244 >> 172.21.15.167:0/1997358244 conn(0x7f9bf40fb290 msgr2=0x7f9bf40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 -- 172.21.15.167:0/1997358244 shutdown_connections 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 -- 172.21.15.167:0/1997358244 wait complete. 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 Processor -- start 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 -- start start 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf40ffcf0 0x7f9bf4103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf4100ef0 0x7f9bf41022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 0x7f9bf4102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4102fb0 con 0x7f9bf40ffcf0 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4103120 con 0x7f9bf4100ef0 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfbe30700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf41a0e10 con 0x7f9bf4101890 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfa3cd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 0x7f9bf4102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:42.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfa3cd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 0x7f9bf4102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:50572/0 (socket says 172.21.15.167:50572) 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.267+0000 7f9bfa3cd700 1 -- 172.21.15.167:0/3678139006 learned_addr learned my addr 172.21.15.167:0/3678139006 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bf93cb700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf4100ef0 0x7f9bf41022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfa3cd700 1 -- 172.21.15.167:0/3678139006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf4100ef0 msgr2=0x7f9bf41022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfa3cd700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf4100ef0 0x7f9bf41022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfa3cd700 1 -- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf40ffcf0 msgr2=0x7f9bf4103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfa3cd700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf40ffcf0 0x7f9bf4103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfa3cd700 1 -- 172.21.15.167:0/3678139006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00095f0 con 0x7f9bf4101890 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfa3cd700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 0x7f9bf4102810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00b970 tx=0x7f9bec00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00c930 con 0x7f9bf4101890 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec014440 con 0x7f9bf4101890 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf41a10a0 con 0x7f9bf4101890 2024-03-22T19:08:42.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0075d0 con 0x7f9bf4101890 2024-03-22T19:08:42.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.268+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf41a1620 con 0x7f9bf4101890 2024-03-22T19:08:42.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.269+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf404ea80 con 0x7f9bf4101890 2024-03-22T19:08:42.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.270+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9bec0145b0 con 0x7f9bf4101890 2024-03-22T19:08:42.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.270+0000 7f9beaffd700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9be00a0b10 0x7f9be00a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:42.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.271+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9bec006e40 con 0x7f9bf4101890 2024-03-22T19:08:42.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.271+0000 7f9bf9bcc700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9be00a0b10 0x7f9be00a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:42.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.271+0000 7f9bf9bcc700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9be00a0b10 0x7f9be00a2fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4100d50 tx=0x7f9be4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:42.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.274+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9bec0954e0 con 0x7f9bf4101890 2024-03-22T19:08:42.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.474+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf40fd490 con 0x7f9be00a0b10 2024-03-22T19:08:42.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.483+0000 7f9beaffd700 1 -- 172.21.15.167:0/3678139006 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f9bf40fd490 con 0x7f9be00a0b10 2024-03-22T19:08:42.484 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9be00a0b10 msgr2=0x7f9be00a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9be00a0b10 0x7f9be00a2fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4100d50 tx=0x7f9be4006d20 comp rx=0 tx=0).stop 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 msgr2=0x7f9bf4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 0x7f9bf4102810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00b970 tx=0x7f9bec00bc80 comp rx=0 tx=0).stop 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 shutdown_connections 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9be00a0b10 0x7f9be00a2fd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4101890 0x7f9bf4102810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf40ffcf0 0x7f9bf4103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 --2- 172.21.15.167:0/3678139006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf4100ef0 0x7f9bf41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:42.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 >> 172.21.15.167:0/3678139006 conn(0x7f9bf40fb290 msgr2=0x7f9bf40fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:42.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 shutdown_connections 2024-03-22T19:08:42.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-03-22T19:08:42.485+0000 7f9bfbe30700 1 -- 172.21.15.167:0/3678139006 wait complete. 2024-03-22T19:08:42.876 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.a.service 2024-03-22T19:08:42.881 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.d.service 2024-03-22T19:08:42.928 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.g.service 2024-03-22T19:08:42.950 INFO:journalctl@ceph.mds.a.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:08:42.954 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.j.service 2024-03-22T19:08:42.993 INFO:journalctl@ceph.mds.d.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:08:42.997 DEBUG:teuthology.orchestra.run.smithi131:mds.b> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.b.service 2024-03-22T19:08:43.000 DEBUG:teuthology.orchestra.run.smithi131:mds.e> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.e.service 2024-03-22T19:08:43.020 INFO:journalctl@ceph.mds.g.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:08:43.039 INFO:journalctl@ceph.mds.b.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:08:43.040 DEBUG:teuthology.orchestra.run.smithi131:mds.h> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.h.service 2024-03-22T19:08:43.061 INFO:journalctl@ceph.mds.j.smithi086.stdout:-- Logs begin at Fri 2024-03-22 18:47:00 UTC. -- 2024-03-22T19:08:43.070 INFO:journalctl@ceph.mds.e.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:08:43.072 DEBUG:teuthology.orchestra.run.smithi131:mds.k> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.k.service 2024-03-22T19:08:43.100 INFO:journalctl@ceph.mds.h.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:08:43.101 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.c.service 2024-03-22T19:08:43.103 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.f.service 2024-03-22T19:08:43.127 INFO:journalctl@ceph.mds.k.smithi131.stdout:-- Logs begin at Fri 2024-03-22 18:40:35 UTC. -- 2024-03-22T19:08:43.134 INFO:journalctl@ceph.mds.c.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:08:43.135 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.i.service 2024-03-22T19:08:43.161 INFO:journalctl@ceph.mds.f.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:08:43.162 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.l.service 2024-03-22T19:08:43.188 INFO:journalctl@ceph.mds.i.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:08:43.189 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-22T19:08:43.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:08:43.215 INFO:journalctl@ceph.mds.l.smithi167.stdout:-- Logs begin at Fri 2024-03-22 18:40:58 UTC. -- 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 -- 172.21.15.86:0/3452499361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc4100c50 msgr2=0x7f0bc4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 --2- 172.21.15.86:0/3452499361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc4100c50 0x7f0bc4101070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40098f0 tx=0x7f0bb4009c00 comp rx=0 tx=0).stop 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 -- 172.21.15.86:0/3452499361 shutdown_connections 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 --2- 172.21.15.86:0/3452499361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc41027e0 0x7f0bc4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 --2- 172.21.15.86:0/3452499361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc4100c50 0x7f0bc4101070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 --2- 172.21.15.86:0/3452499361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4101e40 0x7f0bc41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 -- 172.21.15.86:0/3452499361 >> 172.21.15.86:0/3452499361 conn(0x7f0bc40fc1c0 msgr2=0x7f0bc40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 -- 172.21.15.86:0/3452499361 shutdown_connections 2024-03-22T19:08:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.401+0000 7f0bca400700 1 -- 172.21.15.86:0/3452499361 wait complete. 2024-03-22T19:08:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.402+0000 7f0bca400700 1 Processor -- start 2024-03-22T19:08:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.402+0000 7f0bca400700 1 -- start start 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.402+0000 7f0bca400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4100c50 0x7f0bc4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bca400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 0x7f0bc41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bca400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc41027e0 0x7f0bc41a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bca400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41a09c0 con 0x7f0bc4101e40 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bca400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41a0b00 con 0x7f0bc4100c50 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bca400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41a0c90 con 0x7f0bc41027e0 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 0x7f0bc41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 0x7f0bc41993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60680/0 (socket says 172.21.15.86:60680) 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 -- 172.21.15.86:0/1101486029 learned_addr learned my addr 172.21.15.86:0/1101486029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc3e00700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc41027e0 0x7f0bc41a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc3400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4100c50 0x7f0bc4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 -- 172.21.15.86:0/1101486029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc41027e0 msgr2=0x7f0bc41a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc41027e0 0x7f0bc41a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 -- 172.21.15.86:0/1101486029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4100c50 msgr2=0x7f0bc4198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4100c50 0x7f0bc4198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 -- 172.21.15.86:0/1101486029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb4009520 con 0x7f0bc4101e40 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc3400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4100c50 0x7f0bc4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc3e00700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc41027e0 0x7f0bc41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.403+0000 7f0bc2a00700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 0x7f0bc41993d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000e9a0 tx=0x7f0bb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.404+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0009790 con 0x7f0bc4101e40 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.404+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bb0010430 con 0x7f0bc4101e40 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.404+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc41a0f70 con 0x7f0bc4101e40 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.404+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0018680 con 0x7f0bc4101e40 2024-03-22T19:08:43.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.404+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc41a13f0 con 0x7f0bc4101e40 2024-03-22T19:08:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.405+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc40679e0 con 0x7f0bc4101e40 2024-03-22T19:08:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.407+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0bb00105a0 con 0x7f0bc4101e40 2024-03-22T19:08:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.408+0000 7f0bbbe00700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0ba80a0a90 0x7f0ba80a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.408+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0bb0014070 con 0x7f0bc4101e40 2024-03-22T19:08:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.408+0000 7f0bc3400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0ba80a0a90 0x7f0ba80a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.409+0000 7f0bc3400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0ba80a0a90 0x7f0ba80a2f50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40098c0 tx=0x7f0bb4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:43.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.410+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0bb0095da0 con 0x7f0bc4101e40 2024-03-22T19:08:43.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:43.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:43 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:43.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.660+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bc41a1750 con 0x7f0bc4101e40 2024-03-22T19:08:43.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.661+0000 7f0bbbe00700 1 -- 172.21.15.86:0/1101486029 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f0bb00993c0 con 0x7f0bc4101e40 2024-03-22T19:08:43.662 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:43.662 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:08:35.803767+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:08:43.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0ba80a0a90 msgr2=0x7f0ba80a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0ba80a0a90 0x7f0ba80a2f50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40098c0 tx=0x7f0bb4019040 comp rx=0 tx=0).stop 2024-03-22T19:08:43.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 msgr2=0x7f0bc41993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 0x7f0bc41993d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000e9a0 tx=0x7f0bb000ecb0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 shutdown_connections 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0ba80a0a90 0x7f0ba80a2f50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc41027e0 0x7f0bc41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4100c50 0x7f0bc4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 --2- 172.21.15.86:0/1101486029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bc4101e40 0x7f0bc41993d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 >> 172.21.15.86:0/1101486029 conn(0x7f0bc40fc1c0 msgr2=0x7f0bc41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 shutdown_connections 2024-03-22T19:08:43.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.664+0000 7f0bca400700 1 -- 172.21.15.86:0/1101486029 wait complete. 2024-03-22T19:08:43.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:08:43.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:08:43.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:43.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:43.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:43.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:43 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 -- 172.21.15.86:0/1104050680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd8100d50 msgr2=0x7f6bd8101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 --2- 172.21.15.86:0/1104050680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd8100d50 0x7f6bd8101170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0099c0 tx=0x7f6bcc009cd0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 -- 172.21.15.86:0/1104050680 shutdown_connections 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 --2- 172.21.15.86:0/1104050680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd81028e0 0x7f6bd8109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 --2- 172.21.15.86:0/1104050680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd8100d50 0x7f6bd8101170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 --2- 172.21.15.86:0/1104050680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8101f40 0x7f6bd81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 -- 172.21.15.86:0/1104050680 >> 172.21.15.86:0/1104050680 conn(0x7f6bd80fc2c0 msgr2=0x7f6bd80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 -- 172.21.15.86:0/1104050680 shutdown_connections 2024-03-22T19:08:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.878+0000 7f6be0200700 1 -- 172.21.15.86:0/1104050680 wait complete. 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.879+0000 7f6be0200700 1 Processor -- start 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 -- start start 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8100d50 0x7f6bd8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd8101f40 0x7f6bd8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 0x7f6bd81a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81a0a50 con 0x7f6bd81028e0 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81a0b90 con 0x7f6bd8101f40 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6be0200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81a0d20 con 0x7f6bd8100d50 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bde200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 0x7f6bd81a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bdd800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8100d50 0x7f6bd8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bdd800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8100d50 0x7f6bd8198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:41524/0 (socket says 172.21.15.86:41524) 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bde200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 0x7f6bd81a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60712/0 (socket says 172.21.15.86:60712) 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bdd800700 1 -- 172.21.15.86:0/1762338431 learned_addr learned my addr 172.21.15.86:0/1762338431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bdce00700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd8101f40 0x7f6bd8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bde200700 1 -- 172.21.15.86:0/1762338431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8100d50 msgr2=0x7f6bd8198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bde200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8100d50 0x7f6bd8198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bde200700 1 -- 172.21.15.86:0/1762338431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd8101f40 msgr2=0x7f6bd8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.880+0000 7f6bde200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd8101f40 0x7f6bd8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6bde200700 1 -- 172.21.15.86:0/1762338431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0095f0 con 0x7f6bd81028e0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6bde200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 0x7f6bd81a0480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000efc0 tx=0x7f6bd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0018070 con 0x7f6bd81028e0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bd000f040 con 0x7f6bd81028e0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd81a0fa0 con 0x7f6bd81028e0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000cd40 con 0x7f6bd81028e0 2024-03-22T19:08:43.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.881+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd81a1420 con 0x7f6bd81028e0 2024-03-22T19:08:43.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.882+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd8067ac0 con 0x7f6bd81028e0 2024-03-22T19:08:43.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.884+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6bd0003a00 con 0x7f6bd81028e0 2024-03-22T19:08:43.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.885+0000 7f6bd6000700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6bc00a0a90 0x7f6bc00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:43.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.885+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6bd00cb7b0 con 0x7f6bd81028e0 2024-03-22T19:08:43.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.885+0000 7f6bdd800700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6bc00a0a90 0x7f6bc00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:43.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.886+0000 7f6bdd800700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6bc00a0a90 0x7f6bc00a2f50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc000c00 tx=0x7f6bcc009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:43.889+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6bd0095610 con 0x7f6bd81028e0 2024-03-22T19:08:43.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:43.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:43 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:44.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.095+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bd81a1700 con 0x7f6bd81028e0 2024-03-22T19:08:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.095+0000 7f6bd6000700 1 -- 172.21.15.86:0/1762338431 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bd0098c30 con 0x7f6bd81028e0 2024-03-22T19:08:44.096 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:08:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6bc00a0a90 msgr2=0x7f6bc00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6bc00a0a90 0x7f6bc00a2f50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc000c00 tx=0x7f6bcc009530 comp rx=0 tx=0).stop 2024-03-22T19:08:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 msgr2=0x7f6bd81a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 0x7f6bd81a0480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000efc0 tx=0x7f6bd000c3a0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 shutdown_connections 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6bc00a0a90 0x7f6bc00a2f50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd81028e0 0x7f6bd81a0480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bd8100d50 0x7f6bd8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 --2- 172.21.15.86:0/1762338431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd8101f40 0x7f6bd8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 >> 172.21.15.86:0/1762338431 conn(0x7f6bd80fc2c0 msgr2=0x7f6bd8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 shutdown_connections 2024-03-22T19:08:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.098+0000 7f6be0200700 1 -- 172.21.15.86:0/1762338431 wait complete. 2024-03-22T19:08:44.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 -- 172.21.15.86:0/3006730612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 msgr2=0x7f52fc1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 --2- 172.21.15.86:0/3006730612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc1022a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52e80099c0 tx=0x7f52e8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 -- 172.21.15.86:0/3006730612 shutdown_connections 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 --2- 172.21.15.86:0/3006730612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52fc1027e0 0x7f52fc109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 --2- 172.21.15.86:0/3006730612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc100c50 0x7f52fc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 --2- 172.21.15.86:0/3006730612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc1022a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 -- 172.21.15.86:0/3006730612 >> 172.21.15.86:0/3006730612 conn(0x7f52fc0fc1c0 msgr2=0x7f52fc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 -- 172.21.15.86:0/3006730612 shutdown_connections 2024-03-22T19:08:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.309+0000 7f5303c00700 1 -- 172.21.15.86:0/3006730612 wait complete. 2024-03-22T19:08:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 Processor -- start 2024-03-22T19:08:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 -- start start 2024-03-22T19:08:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52fc100c50 0x7f52fc198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc1027e0 0x7f52fc1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1a08f0 con 0x7f52fc101e40 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1a0a30 con 0x7f52fc100c50 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f5303c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1a0bc0 con 0x7f52fc1027e0 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.310+0000 7f52fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc1027e0 0x7f52fc1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f52fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60730/0 (socket says 172.21.15.86:60730) 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f52fbe00700 1 -- 172.21.15.86:0/3317201218 learned_addr learned my addr 172.21.15.86:0/3317201218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301200700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52fc100c50 0x7f52fc198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 -- 172.21.15.86:0/3317201218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52fc100c50 msgr2=0x7f52fc198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52fc100c50 0x7f52fc198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 -- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 msgr2=0x7f52fc199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 -- 172.21.15.86:0/3317201218 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80095f0 con 0x7f52fc1027e0 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f52fbe00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5301c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc1027e0 0x7f52fc1a0320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52f400efc0 tx=0x7f52f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4018070 con 0x7f52fc1027e0 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52f400f040 con 0x7f52fc1027e0 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc1a0e40 con 0x7f52fc1027e0 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.311+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400cd40 con 0x7f52fc1027e0 2024-03-22T19:08:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.312+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc1a12c0 con 0x7f52fc1027e0 2024-03-22T19:08:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.312+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc0679e0 con 0x7f52fc1027e0 2024-03-22T19:08:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.314+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f52f4003a00 con 0x7f52fc1027e0 2024-03-22T19:08:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.315+0000 7f52f9600700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f52e40a0a90 0x7f52e40a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.316+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52f40cbb50 con 0x7f52fc1027e0 2024-03-22T19:08:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.316+0000 7f5301200700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f52e40a0a90 0x7f52e40a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.316+0000 7f5301200700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f52e40a0a90 0x7f52e40a2f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52fc101ca0 tx=0x7f52f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.318+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f52f40959b0 con 0x7f52fc1027e0 2024-03-22T19:08:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.567+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52fc1a1620 con 0x7f52fc1027e0 2024-03-22T19:08:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.568+0000 7f52f9600700 1 -- 172.21.15.86:0/3317201218 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f52f4098fd0 con 0x7f52fc1027e0 2024-03-22T19:08:44.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:44.568 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f52e40a0a90 msgr2=0x7f52e40a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f52e40a0a90 0x7f52e40a2f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52fc101ca0 tx=0x7f52f0009260 comp rx=0 tx=0).stop 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc1027e0 msgr2=0x7f52fc1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc1027e0 0x7f52fc1a0320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52f400efc0 tx=0x7f52f400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 shutdown_connections 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f52e40a0a90 0x7f52e40a2f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52fc1027e0 0x7f52fc1a0320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52fc100c50 0x7f52fc198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 --2- 172.21.15.86:0/3317201218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52fc101e40 0x7f52fc199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.570+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 >> 172.21.15.86:0/3317201218 conn(0x7f52fc0fc1c0 msgr2=0x7f52fc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:44.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.571+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 shutdown_connections 2024-03-22T19:08:44.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:44.571+0000 7f5303c00700 1 -- 172.21.15.86:0/3317201218 wait complete. 2024-03-22T19:08:44.572 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 1 2024-03-22T19:08:44.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:44 smithi167 ceph-mon[47728]: Deploying daemon mds.l on smithi167 2024-03-22T19:08:44.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:44 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1101486029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:08:44.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:44 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1762338431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:08:44.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:44 smithi086 ceph-mon[38032]: Deploying daemon mds.l on smithi167 2024-03-22T19:08:44.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:44 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1101486029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:08:44.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:44 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1762338431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:08:44.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:44 smithi131 ceph-mon[78559]: Deploying daemon mds.l on smithi167 2024-03-22T19:08:44.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:44 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1101486029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:08:44.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:44 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1762338431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:08:45.492 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:08:45 smithi167 systemd[1]: Started Ceph mds.l for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:45.493 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mds[84359]: -- 172.21.15.167:0/4265977282 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56426171a1a0 con 0x5642616f0800 2024-03-22T19:08:45.493 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l[84336]: starting mds.l at 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3317201218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6832/700195904,v1:172.21.15.167:6833/700195904] up:boot 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: fsmap 1 up:standby 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3317201218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6832/700195904,v1:172.21.15.167:6833/700195904] up:boot 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: fsmap 1 up:standby 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:45.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:45.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:45.912 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:45.912 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3317201218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:45.912 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6832/700195904,v1:172.21.15.167:6833/700195904] up:boot 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: fsmap 1 up:standby 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:45.913 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:46.471 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:08:46 smithi131 systemd[1]: Starting Ceph mds.k for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:46.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:46.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:46 smithi086 ceph-mon[38032]: Deploying daemon mds.k on smithi131 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 -- 172.21.15.86:0/2437099619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1069e0 msgr2=0x7f08ec108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 --2- 172.21.15.86:0/2437099619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1069e0 0x7f08ec108dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b0a0 tx=0x7f08e400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 -- 172.21.15.86:0/2437099619 shutdown_connections 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 --2- 172.21.15.86:0/2437099619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1069e0 0x7f08ec108dd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 --2- 172.21.15.86:0/2437099619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 --2- 172.21.15.86:0/2437099619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1040b0 0x7f08ec1064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 -- 172.21.15.86:0/2437099619 >> 172.21.15.86:0/2437099619 conn(0x7f08ec0fb0d0 msgr2=0x7f08ec0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:46.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.792+0000 7f08f2e00700 1 -- 172.21.15.86:0/2437099619 shutdown_connections 2024-03-22T19:08:46.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.793+0000 7f08f2e00700 1 -- 172.21.15.86:0/2437099619 wait complete. 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.793+0000 7f08f2e00700 1 Processor -- start 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 -- start start 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1040b0 0x7f08ec1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 0x7f08ec1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1a09e0 con 0x7f08ec1069e0 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1a0b20 con 0x7f08ec1040b0 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f2e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1a0cb0 con 0x7f08ec101780 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 0x7f08ec1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08ebe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08eb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1040b0 0x7f08ec1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08ebe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:41552/0 (socket says 172.21.15.86:41552) 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08f0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 0x7f08ec1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60750/0 (socket says 172.21.15.86:60750) 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.794+0000 7f08ebe00700 1 -- 172.21.15.86:0/649089373 learned_addr learned my addr 172.21.15.86:0/649089373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08ebe00700 1 -- 172.21.15.86:0/649089373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1040b0 msgr2=0x7f08ec1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08ebe00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1040b0 0x7f08ec1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08ebe00700 1 -- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 msgr2=0x7f08ec1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08ebe00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 0x7f08ec1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08ebe00700 1 -- 172.21.15.86:0/649089373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8009520 con 0x7f08ec101780 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08f0e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 0x7f08ec1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08eb400700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1040b0 0x7f08ec1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08ebe00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec198eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e9a0 tx=0x7f08dc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc009790 con 0x7f08ec101780 2024-03-22T19:08:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e4009d00 con 0x7f08ec101780 2024-03-22T19:08:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc010430 con 0x7f08ec101780 2024-03-22T19:08:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc018680 con 0x7f08ec101780 2024-03-22T19:08:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.795+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec1a1280 con 0x7f08ec101780 2024-03-22T19:08:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.796+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec108560 con 0x7f08ec101780 2024-03-22T19:08:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.798+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f08dc0105a0 con 0x7f08ec101780 2024-03-22T19:08:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.799+0000 7f08e8c00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f08d00a0a90 0x7f08d00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.799+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08dc014070 con 0x7f08ec101780 2024-03-22T19:08:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.800+0000 7f08eb400700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f08d00a0a90 0x7f08d00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.800+0000 7f08eb400700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f08d00a0a90 0x7f08d00a2f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009fd0 tx=0x7f08d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:46.801+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08dc095e20 con 0x7f08ec101780 2024-03-22T19:08:46.806 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:46 smithi131 ceph-mon[78559]: Deploying daemon mds.k on smithi131 2024-03-22T19:08:46.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:46 smithi167 ceph-mon[47728]: Deploying daemon mds.k on smithi131 2024-03-22T19:08:47.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.071+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08ec067150 con 0x7f08ec101780 2024-03-22T19:08:47.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.072+0000 7f08e8c00700 1 -- 172.21.15.86:0/649089373 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f08dc099440 con 0x7f08ec101780 2024-03-22T19:08:47.073 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:47.073 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-22T19:08:47.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f08d00a0a90 msgr2=0x7f08d00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:47.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f08d00a0a90 0x7f08d00a2f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009fd0 tx=0x7f08d8009260 comp rx=0 tx=0).stop 2024-03-22T19:08:47.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 msgr2=0x7f08ec198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:47.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec198eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e9a0 tx=0x7f08dc00ecb0 comp rx=0 tx=0).stop 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 shutdown_connections 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f08d00a0a90 0x7f08d00a2f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08ec1069e0 0x7f08ec1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec101780 0x7f08ec198eb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 --2- 172.21.15.86:0/649089373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08ec1040b0 0x7f08ec1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 >> 172.21.15.86:0/649089373 conn(0x7f08ec0fb0d0 msgr2=0x7f08ec100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 shutdown_connections 2024-03-22T19:08:47.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:47.075+0000 7f08f2e00700 1 -- 172.21.15.86:0/649089373 wait complete. 2024-03-22T19:08:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-03-22T19:08:47.221 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:08:46 smithi131 ceph-mds[115599]: -- 172.21.15.131:0/3914435972 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653aba861a0 con 0x5653aba5c800 2024-03-22T19:08:47.221 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:08:46 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k[115576]: starting mds.k at 2024-03-22T19:08:47.222 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:08:46 smithi131 bash[115562]: 7637362f14465ba53c3bb21055bc897202032fe6bc3087fd3dbbe0eea9286b1f 2024-03-22T19:08:47.222 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:08:46 smithi131 systemd[1]: Started Ceph mds.k for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:47.498 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:47.498 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:47 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/649089373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:47.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:47.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:47 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/649089373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:47.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:47.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:47 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/649089373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:48.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:48 smithi086 ceph-mon[38032]: Deploying daemon mds.j on smithi086 2024-03-22T19:08:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:48 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6832/2878260168,v1:172.21.15.131:6833/2878260168] up:boot 2024-03-22T19:08:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:48 smithi086 ceph-mon[38032]: fsmap 2 up:standby 2024-03-22T19:08:48.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:48 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-22T19:08:48.759 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:08:48 smithi086 systemd[1]: Starting Ceph mds.j for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:48 smithi167 ceph-mon[47728]: Deploying daemon mds.j on smithi086 2024-03-22T19:08:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:48 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6832/2878260168,v1:172.21.15.131:6833/2878260168] up:boot 2024-03-22T19:08:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:48 smithi167 ceph-mon[47728]: fsmap 2 up:standby 2024-03-22T19:08:48.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:48 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-22T19:08:48.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:48 smithi131 ceph-mon[78559]: Deploying daemon mds.j on smithi086 2024-03-22T19:08:48.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:48 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6832/2878260168,v1:172.21.15.131:6833/2878260168] up:boot 2024-03-22T19:08:48.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:48 smithi131 ceph-mon[78559]: fsmap 2 up:standby 2024-03-22T19:08:48.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:48 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-22T19:08:49.096 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:49.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 -- 172.21.15.86:0/3418686582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8180ff4d0 msgr2=0x7fd8180ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:49.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 --2- 172.21.15.86:0/3418686582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8180ff4d0 0x7fd8180ff8f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0098f0 tx=0x7fd80c009c00 comp rx=0 tx=0).stop 2024-03-22T19:08:49.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 -- 172.21.15.86:0/3418686582 shutdown_connections 2024-03-22T19:08:49.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 --2- 172.21.15.86:0/3418686582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8181007f0 0x7fd818100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 --2- 172.21.15.86:0/3418686582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8180ff4d0 0x7fd8180ff8f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 --2- 172.21.15.86:0/3418686582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ffe30 0x7fd8181002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.307+0000 7fd81f400700 1 -- 172.21.15.86:0/3418686582 >> 172.21.15.86:0/3418686582 conn(0x7fd8180fb0d0 msgr2=0x7fd8180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:49.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.308+0000 7fd81f400700 1 -- 172.21.15.86:0/3418686582 shutdown_connections 2024-03-22T19:08:49.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.308+0000 7fd81f400700 1 -- 172.21.15.86:0/3418686582 wait complete. 2024-03-22T19:08:49.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 Processor -- start 2024-03-22T19:08:49.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 -- start start 2024-03-22T19:08:49.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ff4d0 0x7fd818198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:49.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 0x7fd8181992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8181007f0 0x7fd8181a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8181a08e0 con 0x7fd8180ffe30 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8181a0a20 con 0x7fd8180ff4d0 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd81f400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8181a0bb0 con 0x7fd8181007f0 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd817e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 0x7fd8181992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd817e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 0x7fd8181992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59778/0 (socket says 172.21.15.86:59778) 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.309+0000 7fd817e00700 1 -- 172.21.15.86:0/1545959869 learned_addr learned my addr 172.21.15.86:0/1545959869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd81ca00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ff4d0 0x7fd818198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd81d400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8181007f0 0x7fd8181a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd817e00700 1 -- 172.21.15.86:0/1545959869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8181007f0 msgr2=0x7fd8181a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd817e00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8181007f0 0x7fd8181a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd817e00700 1 -- 172.21.15.86:0/1545959869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ff4d0 msgr2=0x7fd818198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd817e00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ff4d0 0x7fd818198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd817e00700 1 -- 172.21.15.86:0/1545959869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c009520 con 0x7fd8180ffe30 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd81ca00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ff4d0 0x7fd818198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd817e00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 0x7fd8181992f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9a0 tx=0x7fd80400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804009790 con 0x7fd8180ffe30 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd804010430 con 0x7fd8180ffe30 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804018680 con 0x7fd8180ffe30 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.310+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8181a0e90 con 0x7fd8180ffe30 2024-03-22T19:08:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.311+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8181a1310 con 0x7fd8180ffe30 2024-03-22T19:08:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.312+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81810a540 con 0x7fd8180ffe30 2024-03-22T19:08:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.314+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fd8040105a0 con 0x7fd8180ffe30 2024-03-22T19:08:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.315+0000 7fd815600700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd8000a0b10 0x7fd8000a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.315+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd804014070 con 0x7fd8180ffe30 2024-03-22T19:08:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.315+0000 7fd81ca00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd8000a0b10 0x7fd8000a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.316+0000 7fd81ca00700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd8000a0b10 0x7fd8000a2fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005e90 tx=0x7fd80c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:49.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.319+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd8040529f0 con 0x7fd8180ffe30 2024-03-22T19:08:49.508 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:08:49 smithi086 bash[89085]: cb2c14ab8a5acfbf06fabc27bd4678f85bcdfdc5f4c04778b31af04796232c33 2024-03-22T19:08:49.508 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:08:49 smithi086 systemd[1]: Started Ceph mds.j for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:49.509 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mds[89142]: -- 172.21.15.86:0/2065232678 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2177ba1a0 con 0x55b217793000 2024-03-22T19:08:49.509 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-j[89100]: starting mds.j at 2024-03-22T19:08:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.586+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd818067150 con 0x7fd8180ffe30 2024-03-22T19:08:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.586+0000 7fd815600700 1 -- 172.21.15.86:0/1545959869 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7fd804095b40 con 0x7fd8180ffe30 2024-03-22T19:08:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:49.588 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-22T19:08:49.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd8000a0b10 msgr2=0x7fd8000a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:49.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd8000a0b10 0x7fd8000a2fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005e90 tx=0x7fd80c019040 comp rx=0 tx=0).stop 2024-03-22T19:08:49.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 msgr2=0x7fd8181992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 0x7fd8181992f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9a0 tx=0x7fd80400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 shutdown_connections 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd8000a0b10 0x7fd8000a2fd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8181007f0 0x7fd8181a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8180ff4d0 0x7fd818198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 --2- 172.21.15.86:0/1545959869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8180ffe30 0x7fd8181992f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 >> 172.21.15.86:0/1545959869 conn(0x7fd8180fb0d0 msgr2=0x7fd8180fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 shutdown_connections 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:49.589+0000 7fd81f400700 1 -- 172.21.15.86:0/1545959869 wait complete. 2024-03-22T19:08:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:49.938 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:49.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:49.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:49.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:50.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:50 smithi086 ceph-mon[38032]: Deploying daemon mds.d on smithi086 2024-03-22T19:08:50.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:50 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6834/1672168335,v1:172.21.15.86:6835/1672168335] up:boot 2024-03-22T19:08:50.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:50 smithi086 ceph-mon[38032]: fsmap 3 up:standby 2024-03-22T19:08:50.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-22T19:08:50.587 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:50 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1545959869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:50.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:50 smithi167 ceph-mon[47728]: Deploying daemon mds.d on smithi086 2024-03-22T19:08:50.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:50 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6834/1672168335,v1:172.21.15.86:6835/1672168335] up:boot 2024-03-22T19:08:50.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:50 smithi167 ceph-mon[47728]: fsmap 3 up:standby 2024-03-22T19:08:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-22T19:08:50.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:50 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1545959869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:50.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:50 smithi131 ceph-mon[78559]: Deploying daemon mds.d on smithi086 2024-03-22T19:08:50.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:50 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6834/1672168335,v1:172.21.15.86:6835/1672168335] up:boot 2024-03-22T19:08:50.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:50 smithi131 ceph-mon[78559]: fsmap 3 up:standby 2024-03-22T19:08:50.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-22T19:08:50.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:50 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1545959869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:51.259 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:08:51 smithi086 systemd[1]: Starting Ceph mds.d for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:51.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:51 smithi167 ceph-mon[47728]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.826+0000 7ff86fc00700 1 -- 172.21.15.86:0/464426478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868101e40 msgr2=0x7ff8681022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.826+0000 7ff86fc00700 1 --2- 172.21.15.86:0/464426478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868101e40 0x7ff8681022a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8540099c0 tx=0x7ff854009cd0 comp rx=0 tx=0).stop 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 -- 172.21.15.86:0/464426478 shutdown_connections 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 --2- 172.21.15.86:0/464426478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8681027e0 0x7ff868109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 --2- 172.21.15.86:0/464426478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff868100c50 0x7ff868101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 --2- 172.21.15.86:0/464426478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868101e40 0x7ff8681022a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 -- 172.21.15.86:0/464426478 >> 172.21.15.86:0/464426478 conn(0x7ff8680fc1c0 msgr2=0x7ff8680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 -- 172.21.15.86:0/464426478 shutdown_connections 2024-03-22T19:08:51.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.827+0000 7ff86fc00700 1 -- 172.21.15.86:0/464426478 wait complete. 2024-03-22T19:08:51.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 Processor -- start 2024-03-22T19:08:51.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 -- start start 2024-03-22T19:08:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 0x7ff868198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff868101e40 0x7ff8681992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8681027e0 0x7ff8681a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8681a0860 con 0x7ff868100c50 2024-03-22T19:08:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8681a09d0 con 0x7ff868101e40 2024-03-22T19:08:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86fc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8681a0b90 con 0x7ff8681027e0 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 0x7ff868198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.828+0000 7ff86d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 0x7ff868198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59828/0 (socket says 172.21.15.86:59828) 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 -- 172.21.15.86:0/2123950009 learned_addr learned my addr 172.21.15.86:0/2123950009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff867e00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff868101e40 0x7ff8681992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86dc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8681027e0 0x7ff8681a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 -- 172.21.15.86:0/2123950009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8681027e0 msgr2=0x7ff8681a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8681027e0 0x7ff8681a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 -- 172.21.15.86:0/2123950009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff868101e40 msgr2=0x7ff8681992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff868101e40 0x7ff8681992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 -- 172.21.15.86:0/2123950009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540095f0 con 0x7ff868100c50 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86d200700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 0x7ff868198d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e970 tx=0x7ff85c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c009790 con 0x7ff868100c50 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff85c00c7e0 con 0x7ff868100c50 2024-03-22T19:08:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8681a0e70 con 0x7ff868100c50 2024-03-22T19:08:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.829+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c003c70 con 0x7ff868100c50 2024-03-22T19:08:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.830+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8681a1290 con 0x7ff868100c50 2024-03-22T19:08:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.831+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8680679e0 con 0x7ff868100c50 2024-03-22T19:08:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.832+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff85c00cc50 con 0x7ff868100c50 2024-03-22T19:08:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.834+0000 7ff865600700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8500a0a90 0x7ff8500a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.834+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff85c014070 con 0x7ff868100c50 2024-03-22T19:08:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.834+0000 7ff867e00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8500a0a90 0x7ff8500a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.834+0000 7ff867e00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8500a0a90 0x7ff8500a2f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff854009fd0 tx=0x7ff85400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:51.840+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff85c095bd0 con 0x7ff868100c50 2024-03-22T19:08:51.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:51 smithi131 ceph-mon[78559]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:52.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:51 smithi086 ceph-mon[38032]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:52.009 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:08:51 smithi086 ceph-mds[89514]: -- 172.21.15.86:0/3501725033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55563d2d41a0 con 0x55563d2acc00 2024-03-22T19:08:52.009 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:08:51 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-d[89489]: starting mds.d at 2024-03-22T19:08:52.009 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:08:51 smithi086 bash[89474]: 1cc41c54c2fdb1c7d2bedd7d5ffa685f0a4be3597bdb5a3f4f814f2e61425771 2024-03-22T19:08:52.009 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:08:51 smithi086 systemd[1]: Started Ceph mds.d for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.095+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8681a1900 con 0x7ff868100c50 2024-03-22T19:08:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.096+0000 7ff865600700 1 -- 172.21.15.86:0/2123950009 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7ff85c003dd0 con 0x7ff868100c50 2024-03-22T19:08:52.098 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:52.098 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-22T19:08:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.098+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8500a0a90 msgr2=0x7ff8500a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8500a0a90 0x7ff8500a2f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff854009fd0 tx=0x7ff85400b540 comp rx=0 tx=0).stop 2024-03-22T19:08:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 msgr2=0x7ff868198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 0x7ff868198d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e970 tx=0x7ff85c00ec80 comp rx=0 tx=0).stop 2024-03-22T19:08:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 shutdown_connections 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8500a0a90 0x7ff8500a2f50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8681027e0 0x7ff8681a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff868100c50 0x7ff868198d70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 --2- 172.21.15.86:0/2123950009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff868101e40 0x7ff8681992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 >> 172.21.15.86:0/2123950009 conn(0x7ff8680fc1c0 msgr2=0x7ff868072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 shutdown_connections 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:52.099+0000 7ff86fc00700 1 -- 172.21.15.86:0/2123950009 wait complete. 2024-03-22T19:08:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-03-22T19:08:52.894 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:52.894 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: Deploying daemon mds.e on smithi131 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2123950009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6836/3289005201,v1:172.21.15.86:6837/3289005201] up:boot 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: fsmap 4 up:standby 2024-03-22T19:08:52.895 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:52 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: Deploying daemon mds.e on smithi131 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2123950009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6836/3289005201,v1:172.21.15.86:6837/3289005201] up:boot 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: fsmap 4 up:standby 2024-03-22T19:08:53.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:52 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-22T19:08:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:53.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: Deploying daemon mds.e on smithi131 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2123950009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6836/3289005201,v1:172.21.15.86:6837/3289005201] up:boot 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: fsmap 4 up:standby 2024-03-22T19:08:53.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:52 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-22T19:08:53.222 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:08:53 smithi131 systemd[1]: Starting Ceph mds.e for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:53.721 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:08:53 smithi131 ceph-mds[115942]: -- 172.21.15.131:0/3779708273 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e09e44a1a0 con 0x55e09e420c00 2024-03-22T19:08:53.721 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:08:53 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e[115919]: starting mds.e at 2024-03-22T19:08:53.722 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:08:53 smithi131 bash[115904]: 1877d9779774a8a68e1482a4677de2ce0403dcb4ebf8c33b00f9beb1587f4c49 2024-03-22T19:08:53.722 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:08:53 smithi131 systemd[1]: Started Ceph mds.e for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:54.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:54.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 -- 172.21.15.86:0/359535051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 msgr2=0x7f037810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:54.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 --2- 172.21.15.86:0/359535051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f037810afa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0370009990 tx=0x7f0370009ca0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 -- 172.21.15.86:0/359535051 shutdown_connections 2024-03-22T19:08:54.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 --2- 172.21.15.86:0/359535051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f037810afa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 --2- 172.21.15.86:0/359535051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0378074df0 0x7f0378073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 --2- 172.21.15.86:0/359535051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378073810 0x7f0378073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 -- 172.21.15.86:0/359535051 >> 172.21.15.86:0/359535051 conn(0x7f03780fc1c0 msgr2=0x7f03780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 -- 172.21.15.86:0/359535051 shutdown_connections 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.321+0000 7f037e600700 1 -- 172.21.15.86:0/359535051 wait complete. 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.322+0000 7f037e600700 1 Processor -- start 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 -- start start 2024-03-22T19:08:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0378073810 0x7f0378198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 0x7f03781992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f03781a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03781a0850 con 0x7f0378074df0 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03781a09c0 con 0x7f0378108b10 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f037e600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03781a0b80 con 0x7f0378073810 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f036fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 0x7f03781992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f036fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 0x7f03781992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59836/0 (socket says 172.21.15.86:59836) 2024-03-22T19:08:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f036fe00700 1 -- 172.21.15.86:0/634919405 learned_addr learned my addr 172.21.15.86:0/634919405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f0377e00700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f03781a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f0377400700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0378073810 0x7f0378198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.323+0000 7f036fe00700 1 -- 172.21.15.86:0/634919405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0378073810 msgr2=0x7f0378198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f036fe00700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0378073810 0x7f0378198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f036fe00700 1 -- 172.21.15.86:0/634919405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 msgr2=0x7f03781a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f036fe00700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f03781a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f036fe00700 1 -- 172.21.15.86:0/634919405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009520 con 0x7f0378074df0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f0377e00700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f03781a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f036fe00700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 0x7f03781992a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f036000ea50 tx=0x7f036000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000cbd0 con 0x7f0378074df0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f036000cd30 con 0x7f0378074df0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03700095f0 con 0x7f0378074df0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360018680 con 0x7f0378074df0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.324+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781a1190 con 0x7f0378074df0 2024-03-22T19:08:54.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.325+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03781078d0 con 0x7f0378074df0 2024-03-22T19:08:54.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.327+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0360010430 con 0x7f0378074df0 2024-03-22T19:08:54.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.328+0000 7f0374c00700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f035c0a0a90 0x7f035c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:54.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.328+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0360014070 con 0x7f0378074df0 2024-03-22T19:08:54.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.329+0000 7f0377400700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f035c0a0a90 0x7f035c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:54.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.329+0000 7f0377400700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f035c0a0a90 0x7f035c0a2f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f037819a1e0 tx=0x7f036800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:54.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.332+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0360095ec0 con 0x7f0378074df0 2024-03-22T19:08:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: Deploying daemon mds.f on smithi167 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6834/4106390239,v1:172.21.15.131:6835/4106390239] up:boot 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: fsmap 5 up:standby 2024-03-22T19:08:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-22T19:08:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.584+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03780679e0 con 0x7f0378074df0 2024-03-22T19:08:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.585+0000 7f0374c00700 1 -- 172.21.15.86:0/634919405 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4291 (secure 0 0 0) 0x7f03600b9070 con 0x7f0378074df0 2024-03-22T19:08:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:54.586 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.587+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f035c0a0a90 msgr2=0x7f035c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.587+0000 7f037e600700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f035c0a0a90 0x7f035c0a2f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f037819a1e0 tx=0x7f036800b040 comp rx=0 tx=0).stop 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.587+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 msgr2=0x7f03781992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.587+0000 7f037e600700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 0x7f03781992a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f036000ea50 tx=0x7f036000ed60 comp rx=0 tx=0).stop 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.587+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 shutdown_connections 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.587+0000 7f037e600700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f035c0a0a90 0x7f035c0a2f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.588+0000 7f037e600700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0378108b10 0x7f03781a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.588+0000 7f037e600700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0378073810 0x7f0378198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.588+0000 7f037e600700 1 --2- 172.21.15.86:0/634919405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0378074df0 0x7f03781992a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.588+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 >> 172.21.15.86:0/634919405 conn(0x7f03780fc1c0 msgr2=0x7f0378104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.588+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 shutdown_connections 2024-03-22T19:08:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:54.588+0000 7f037e600700 1 -- 172.21.15.86:0/634919405 wait complete. 2024-03-22T19:08:54.590 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-03-22T19:08:54.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: Deploying daemon mds.f on smithi167 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6834/4106390239,v1:172.21.15.131:6835/4106390239] up:boot 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: fsmap 5 up:standby 2024-03-22T19:08:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-22T19:08:54.823 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:08:54 smithi167 systemd[1]: Starting Ceph mds.f for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:54.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: Deploying daemon mds.f on smithi167 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6834/4106390239,v1:172.21.15.131:6835/4106390239] up:boot 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: fsmap 5 up:standby 2024-03-22T19:08:54.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-22T19:08:55.501 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mds[84699]: -- 172.21.15.167:0/3852437161 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f40465e1a0 con 0x55f404634800 2024-03-22T19:08:55.502 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f[84676]: starting mds.f at 2024-03-22T19:08:55.502 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:08:55 smithi167 bash[84661]: 8d33218c115e3013bfbaa79d71aacf7dc08899670fcc900df687d210bdbfd1e3 2024-03-22T19:08:55.502 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:08:55 smithi167 systemd[1]: Started Ceph mds.f for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:55.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/634919405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:55 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/634919405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:55 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:55.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:55.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/634919405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:55.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:55.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:55.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:55.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:55 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:56.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:56.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:56 smithi167 ceph-mon[47728]: Deploying daemon mds.c on smithi167 2024-03-22T19:08:56.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:56 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] up:boot 2024-03-22T19:08:56.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:56 smithi167 ceph-mon[47728]: fsmap 6 up:standby 2024-03-22T19:08:56.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:56 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-22T19:08:56.741 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:08:56 smithi167 systemd[1]: Starting Ceph mds.c for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:56.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:56 smithi086 ceph-mon[38032]: Deploying daemon mds.c on smithi167 2024-03-22T19:08:56.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:56 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] up:boot 2024-03-22T19:08:56.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:56 smithi086 ceph-mon[38032]: fsmap 6 up:standby 2024-03-22T19:08:56.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:56 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-22T19:08:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 -- 172.21.15.86:0/3189023825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 msgr2=0x7f35b810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 --2- 172.21.15.86:0/3189023825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b810afa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b0a0 tx=0x7f35b000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:08:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 -- 172.21.15.86:0/3189023825 shutdown_connections 2024-03-22T19:08:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 --2- 172.21.15.86:0/3189023825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b810afa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 --2- 172.21.15.86:0/3189023825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b8074df0 0x7f35b8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 --2- 172.21.15.86:0/3189023825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.808+0000 7f35bea00700 1 -- 172.21.15.86:0/3189023825 >> 172.21.15.86:0/3189023825 conn(0x7f35b80fc1c0 msgr2=0x7f35b80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.809+0000 7f35bea00700 1 -- 172.21.15.86:0/3189023825 shutdown_connections 2024-03-22T19:08:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.809+0000 7f35bea00700 1 -- 172.21.15.86:0/3189023825 wait complete. 2024-03-22T19:08:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 Processor -- start 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 -- start start 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b8074df0 0x7f35b8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b81a0a20 con 0x7f35b8108b10 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b81a0b60 con 0x7f35b8074df0 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bea00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b81a0cf0 con 0x7f35b8073810 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35b7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35bca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35b7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36896/0 (socket says 172.21.15.86:36896) 2024-03-22T19:08:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35b7e00700 1 -- 172.21.15.86:0/1619934897 learned_addr learned my addr 172.21.15.86:0/1619934897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.810+0000 7f35b7400700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b8074df0 0x7f35b8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b7e00700 1 -- 172.21.15.86:0/1619934897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b8074df0 msgr2=0x7f35b8199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b7e00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b8074df0 0x7f35b8199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b7e00700 1 -- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 msgr2=0x7f35b81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b7e00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b7e00700 1 -- 172.21.15.86:0/1619934897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0009d00 con 0x7f35b8073810 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35bca00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b7e00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8198ef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35a800e9a0 tx=0x7f35a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8009790 con 0x7f35b8073810 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a8010430 con 0x7f35b8073810 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b81a0fd0 con 0x7f35b8073810 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8018680 con 0x7f35b8073810 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.811+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b81a1400 con 0x7f35b8073810 2024-03-22T19:08:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.812+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b81078d0 con 0x7f35b8073810 2024-03-22T19:08:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.814+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f35a80105a0 con 0x7f35b8073810 2024-03-22T19:08:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.815+0000 7f35b4c00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f359c0a0b10 0x7f359c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.815+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35a8014070 con 0x7f35b8073810 2024-03-22T19:08:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.815+0000 7f35b7400700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f359c0a0b10 0x7f359c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.816+0000 7f35b7400700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f359c0a0b10 0x7f359c0a2fd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35b8074b10 tx=0x7f35a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:56.819+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f35a80529f0 con 0x7f35b8073810 2024-03-22T19:08:56.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:56 smithi131 ceph-mon[78559]: Deploying daemon mds.c on smithi167 2024-03-22T19:08:56.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:56 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] up:boot 2024-03-22T19:08:56.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:56 smithi131 ceph-mon[78559]: fsmap 6 up:standby 2024-03-22T19:08:56.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:56 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-22T19:08:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.075+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35b80679e0 con 0x7f35b8073810 2024-03-22T19:08:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.075+0000 7f35b4c00700 1 -- 172.21.15.86:0/1619934897 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5051 (secure 0 0 0) 0x7f35a8095b40 con 0x7f35b8073810 2024-03-22T19:08:57.077 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:57.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-03-22T19:08:57.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f359c0a0b10 msgr2=0x7f359c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:57.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f359c0a0b10 0x7f359c0a2fd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35b8074b10 tx=0x7f35a4009260 comp rx=0 tx=0).stop 2024-03-22T19:08:57.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 msgr2=0x7f35b8198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8198ef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35a800e9a0 tx=0x7f35a800ecb0 comp rx=0 tx=0).stop 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 shutdown_connections 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f359c0a0b10 0x7f359c0a2fd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b8108b10 0x7f35b81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35b8073810 0x7f35b8198ef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 --2- 172.21.15.86:0/1619934897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b8074df0 0x7f35b8199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 >> 172.21.15.86:0/1619934897 conn(0x7f35b80fc1c0 msgr2=0x7f35b8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 shutdown_connections 2024-03-22T19:08:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:57.078+0000 7f35bea00700 1 -- 172.21.15.86:0/1619934897 wait complete. 2024-03-22T19:08:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-03-22T19:08:57.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:08:56 smithi167 bash[85004]: e0a637683c294a657a7edab5b9ee9a581607d292bc5b512377da30842164c9f6 2024-03-22T19:08:57.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:08:56 smithi167 systemd[1]: Started Ceph mds.c for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:57.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:08:56 smithi167 ceph-mds[85042]: -- 172.21.15.167:0/3951897470 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627f2e161a0 con 0x5627f2dec800 2024-03-22T19:08:57.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:08:56 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c[85019]: starting mds.c at 2024-03-22T19:08:57.710 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:57.710 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:57.710 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:57.711 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:57.711 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:57.711 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:57.711 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:57.711 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:57 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1619934897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:57 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1619934897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:58.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:58.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:57 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1619934897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:08:58.520 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:08:58 smithi131 systemd[1]: Starting Ceph mds.h for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:08:58.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:58 smithi167 ceph-mon[47728]: Deploying daemon mds.h on smithi131 2024-03-22T19:08:58.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:58 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6836/1695283976,v1:172.21.15.167:6837/1695283976] up:boot 2024-03-22T19:08:58.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:58 smithi167 ceph-mon[47728]: fsmap 7 up:standby 2024-03-22T19:08:58.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:58 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-22T19:08:58.971 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:08:58 smithi131 ceph-mds[116284]: -- 172.21.15.131:0/815133526 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561dfd2da1a0 con 0x561dfd2b0800 2024-03-22T19:08:58.971 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:08:58 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h[116261]: starting mds.h at 2024-03-22T19:08:58.971 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:08:58 smithi131 bash[116246]: 4ab593a95ddaeb508906744af0d92ba7dbdec4cabf02ae35d806db342a8b58f0 2024-03-22T19:08:58.971 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:08:58 smithi131 systemd[1]: Started Ceph mds.h for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:08:58.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:58 smithi131 ceph-mon[78559]: Deploying daemon mds.h on smithi131 2024-03-22T19:08:58.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:58 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6836/1695283976,v1:172.21.15.167:6837/1695283976] up:boot 2024-03-22T19:08:58.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:58 smithi131 ceph-mon[78559]: fsmap 7 up:standby 2024-03-22T19:08:58.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:58 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-22T19:08:59.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:58 smithi086 ceph-mon[38032]: Deploying daemon mds.h on smithi131 2024-03-22T19:08:59.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:58 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6836/1695283976,v1:172.21.15.167:6837/1695283976] up:boot 2024-03-22T19:08:59.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:58 smithi086 ceph-mon[38032]: fsmap 7 up:standby 2024-03-22T19:08:59.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:58 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-22T19:08:59.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:08:59.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.300+0000 7f8200400700 1 -- 172.21.15.86:0/3378494672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f8101e40 msgr2=0x7f81f81022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.300+0000 7f8200400700 1 --2- 172.21.15.86:0/3378494672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f8101e40 0x7f81f81022a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f81e40099c0 tx=0x7f81e4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 -- 172.21.15.86:0/3378494672 shutdown_connections 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 --2- 172.21.15.86:0/3378494672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f81027e0 0x7f81f8109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 --2- 172.21.15.86:0/3378494672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 --2- 172.21.15.86:0/3378494672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f8101e40 0x7f81f81022a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 -- 172.21.15.86:0/3378494672 >> 172.21.15.86:0/3378494672 conn(0x7f81f80fc1c0 msgr2=0x7f81f80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 -- 172.21.15.86:0/3378494672 shutdown_connections 2024-03-22T19:08:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.301+0000 7f8200400700 1 -- 172.21.15.86:0/3378494672 wait complete. 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 Processor -- start 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 -- start start 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f8101e40 0x7f81f81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f81027e0 0x7f81f81a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81a0890 con 0x7f81f81027e0 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81a0a00 con 0x7f81f8101e40 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f8200400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81a0bc0 con 0x7f81f8100c50 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f81fda00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f81fda00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:33498/0 (socket says 172.21.15.86:33498) 2024-03-22T19:08:59.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f81fda00700 1 -- 172.21.15.86:0/4040546000 learned_addr learned my addr 172.21.15.86:0/4040546000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.302+0000 7f81fe400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f81027e0 0x7f81f81a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fd000700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f8101e40 0x7f81f81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fda00700 1 -- 172.21.15.86:0/4040546000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f8101e40 msgr2=0x7f81f81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fda00700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f8101e40 0x7f81f81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fda00700 1 -- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f81027e0 msgr2=0x7f81f81a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fda00700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f81027e0 0x7f81f81a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fda00700 1 -- 172.21.15.86:0/4040546000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40095f0 con 0x7f81f8100c50 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fd000700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f8101e40 0x7f81f81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81fda00700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8198da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00e970 tx=0x7f81ec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec009790 con 0x7f81f8100c50 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81ec00eec0 con 0x7f81f8100c50 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f81a0ea0 con 0x7f81f8100c50 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec003cf0 con 0x7f81f8100c50 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.303+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f81a12c0 con 0x7f81f8100c50 2024-03-22T19:08:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.304+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f80679e0 con 0x7f81f8100c50 2024-03-22T19:08:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.306+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f81ec00ccd0 con 0x7f81f8100c50 2024-03-22T19:08:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.308+0000 7f81f6000700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81e00a0a90 0x7f81e00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:08:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.308+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81ec014070 con 0x7f81f8100c50 2024-03-22T19:08:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.308+0000 7f81fd000700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81e00a0a90 0x7f81e00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:08:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.308+0000 7f81fd000700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81e00a0a90 0x7f81e00a2f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b580 tx=0x7f81e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:08:59.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.310+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f81ec095fa0 con 0x7f81f8100c50 2024-03-22T19:08:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.565+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81f81a1930 con 0x7f81f8100c50 2024-03-22T19:08:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.565+0000 7f81f6000700 1 -- 172.21.15.86:0/4040546000 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6571 (secure 0 0 0) 0x7f81ec0995c0 con 0x7f81f8100c50 2024-03-22T19:08:59.567 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:08:59.567 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81e00a0a90 msgr2=0x7f81e00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81e00a0a90 0x7f81e00a2f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b580 tx=0x7f81e401a040 comp rx=0 tx=0).stop 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 msgr2=0x7f81f8198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8198da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00e970 tx=0x7f81ec00ec80 comp rx=0 tx=0).stop 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 shutdown_connections 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81e00a0a90 0x7f81e00a2f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f81027e0 0x7f81f81a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f8100c50 0x7f81f8198da0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 --2- 172.21.15.86:0/4040546000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f8101e40 0x7f81f81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:08:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 >> 172.21.15.86:0/4040546000 conn(0x7f81f80fc1c0 msgr2=0x7f81f8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:08:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 shutdown_connections 2024-03-22T19:08:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:08:59.568+0000 7f8200400700 1 -- 172.21.15.86:0/4040546000 wait complete. 2024-03-22T19:08:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T19:08:59.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:59.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:59.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:59.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:08:59 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:08:59.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:08:59.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:59.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:59.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:08:59.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:08:59.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:08:59.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:08:59 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:00.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:00.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:00.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:00.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:00.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:00.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:00.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:08:59 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:00.329 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:09:00 smithi167 systemd[1]: Starting Ceph mds.i for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:09:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-mon[47728]: Deploying daemon mds.i on smithi167 2024-03-22T19:09:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] up:boot 2024-03-22T19:09:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-mon[47728]: fsmap 8 up:standby 2024-03-22T19:09:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-22T19:09:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/4040546000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:00.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-mds[85382]: -- 172.21.15.167:0/546353742 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558acfd5e1a0 con 0x558acfd35000 2024-03-22T19:09:00.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:09:00 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i[85359]: starting mds.i at 2024-03-22T19:09:00.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:09:00 smithi167 bash[85344]: 8661c4f0c14c5459dd9a53ca28bfb783194f38323d5703f0210c06abf9377bfa 2024-03-22T19:09:00.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:09:00 smithi167 systemd[1]: Started Ceph mds.i for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:09:00.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:00 smithi131 ceph-mon[78559]: Deploying daemon mds.i on smithi167 2024-03-22T19:09:00.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:00 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] up:boot 2024-03-22T19:09:00.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:00 smithi131 ceph-mon[78559]: fsmap 8 up:standby 2024-03-22T19:09:00.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-22T19:09:00.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:00 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/4040546000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:01.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:00 smithi086 ceph-mon[38032]: Deploying daemon mds.i on smithi167 2024-03-22T19:09:01.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:00 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] up:boot 2024-03-22T19:09:01.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:00 smithi086 ceph-mon[38032]: fsmap 8 up:standby 2024-03-22T19:09:01.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-22T19:09:01.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:00 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/4040546000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:01.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:09:01.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:01.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:01.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:01.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:01 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 -- 172.21.15.86:0/3401895048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18501026b0 msgr2=0x7f1850108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 --2- 172.21.15.86:0/3401895048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18501026b0 0x7f1850108f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f184800b0d0 tx=0x7f184800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 -- 172.21.15.86:0/3401895048 shutdown_connections 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 --2- 172.21.15.86:0/3401895048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18501026b0 0x7f1850108f60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 --2- 172.21.15.86:0/3401895048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1850100b20 0x7f1850100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 --2- 172.21.15.86:0/3401895048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 0x7f1850102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 -- 172.21.15.86:0/3401895048 >> 172.21.15.86:0/3401895048 conn(0x7f18500fc090 msgr2=0x7f18500fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 -- 172.21.15.86:0/3401895048 shutdown_connections 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.788+0000 7f184fe00700 1 -- 172.21.15.86:0/3401895048 wait complete. 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.789+0000 7f184fe00700 1 Processor -- start 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.789+0000 7f184fe00700 1 -- start start 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 0x7f1850196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 0x7f18501970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18501026b0 0x7f185019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019e6c0 con 0x7f1850100b20 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019e800 con 0x7f18501026b0 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184fe00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019e990 con 0x7f1850101d10 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 0x7f1850196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 0x7f1850196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39558/0 (socket says 172.21.15.86:39558) 2024-03-22T19:09:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 -- 172.21.15.86:0/3234259599 learned_addr learned my addr 172.21.15.86:0/3234259599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184e000700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 0x7f18501970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184f400700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18501026b0 0x7f185019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 -- 172.21.15.86:0/3234259599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 msgr2=0x7f18501970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 0x7f18501970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 -- 172.21.15.86:0/3234259599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18501026b0 msgr2=0x7f185019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18501026b0 0x7f185019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184ea00700 1 -- 172.21.15.86:0/3234259599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840009520 con 0x7f1850100b20 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.790+0000 7f184e000700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 0x7f18501970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.791+0000 7f184ea00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 0x7f1850196b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f184000ea50 tx=0x7f184000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.791+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184000cbd0 con 0x7f1850100b20 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.791+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f184000cd30 con 0x7f1850100b20 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.791+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18400186c0 con 0x7f1850100b20 2024-03-22T19:09:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.791+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1848009d00 con 0x7f1850100b20 2024-03-22T19:09:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.791+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185019eff0 con 0x7f1850100b20 2024-03-22T19:09:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.792+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1850107ed0 con 0x7f1850100b20 2024-03-22T19:09:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.794+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f1840018820 con 0x7f1850100b20 2024-03-22T19:09:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.795+0000 7f1847400700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18340a0b10 0x7f18340a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.795+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1840014070 con 0x7f1850100b20 2024-03-22T19:09:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.795+0000 7f184e000700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18340a0b10 0x7f18340a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.796+0000 7f184e000700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18340a0b10 0x7f18340a2fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f183c009cb0 tx=0x7f183c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:01.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:01.799+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1840095ee0 con 0x7f1850100b20 2024-03-22T19:09:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:01.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:01 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:01 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:02.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.057+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18500670e0 con 0x7f1850100b20 2024-03-22T19:09:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.057+0000 7f1847400700 1 -- 172.21.15.86:0/3234259599 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7333 (secure 0 0 0) 0x7f1840099500 con 0x7f1850100b20 2024-03-22T19:09:02.059 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:02.059 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-22T19:09:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18340a0b10 msgr2=0x7f18340a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18340a0b10 0x7f18340a2fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f183c009cb0 tx=0x7f183c009260 comp rx=0 tx=0).stop 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 msgr2=0x7f1850196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 0x7f1850196b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f184000ea50 tx=0x7f184000ed60 comp rx=0 tx=0).stop 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 shutdown_connections 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18340a0b10 0x7f18340a2fd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18501026b0 0x7f185019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850100b20 0x7f1850196b90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 --2- 172.21.15.86:0/3234259599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1850101d10 0x7f18501970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 >> 172.21.15.86:0/3234259599 conn(0x7f18500fc090 msgr2=0x7f1850105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 shutdown_connections 2024-03-22T19:09:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:02.060+0000 7f184fe00700 1 -- 172.21.15.86:0/3234259599 wait complete. 2024-03-22T19:09:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-03-22T19:09:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:02 smithi086 ceph-mon[38032]: Deploying daemon mds.g on smithi086 2024-03-22T19:09:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:02 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] up:boot 2024-03-22T19:09:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:02 smithi086 ceph-mon[38032]: fsmap 9 up:standby 2024-03-22T19:09:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:02 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-22T19:09:02.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:02 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3234259599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:02.760 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:09:02 smithi086 systemd[1]: Starting Ceph mds.g for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:09:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:02 smithi167 ceph-mon[47728]: Deploying daemon mds.g on smithi086 2024-03-22T19:09:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:02 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] up:boot 2024-03-22T19:09:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:02 smithi167 ceph-mon[47728]: fsmap 9 up:standby 2024-03-22T19:09:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:02 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-22T19:09:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:02 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3234259599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:02.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:02 smithi131 ceph-mon[78559]: Deploying daemon mds.g on smithi086 2024-03-22T19:09:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:02 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] up:boot 2024-03-22T19:09:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:02 smithi131 ceph-mon[78559]: fsmap 9 up:standby 2024-03-22T19:09:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:02 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-22T19:09:02.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:02 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3234259599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:03.508 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mds[90084]: -- 172.21.15.86:0/850310773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e10aa521a0 con 0x55e10aa2a800 2024-03-22T19:09:03.509 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g[90061]: starting mds.g at 2024-03-22T19:09:03.509 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:09:03 smithi086 bash[90046]: d2801444548f78ecff21715608f13888b041b87564484ab79636bd7e5cd936e8 2024-03-22T19:09:03.509 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:09:03 smithi086 systemd[1]: Started Ceph mds.g for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:03 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:03.851 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:03 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:03.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:03.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:03.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:03.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:03.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:03 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:04.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- 172.21.15.86:0/1054034500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764100c50 msgr2=0x7f1764101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- 172.21.15.86:0/1054034500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764100c50 0x7f1764101070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f17540099c0 tx=0x7f1754009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- 172.21.15.86:0/1054034500 shutdown_connections 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- 172.21.15.86:0/1054034500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17641027e0 0x7f1764109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- 172.21.15.86:0/1054034500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764100c50 0x7f1764101070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- 172.21.15.86:0/1054034500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1764101e40 0x7f17641022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- 172.21.15.86:0/1054034500 >> 172.21.15.86:0/1054034500 conn(0x7f17640fc1c0 msgr2=0x7f17640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- 172.21.15.86:0/1054034500 shutdown_connections 2024-03-22T19:09:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- 172.21.15.86:0/1054034500 wait complete. 2024-03-22T19:09:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 Processor -- start 2024-03-22T19:09:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- start start 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1764100c50 0x7f1764196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 0x7f1764197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17641027e0 0x7f176419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176419e700 con 0x7f1764101e40 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176419e840 con 0x7f1764100c50 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176419e9d0 con 0x7f17641027e0 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 0x7f1764197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 0x7f1764197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39580/0 (socket says 172.21.15.86:39580) 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 -- 172.21.15.86:0/3576681712 learned_addr learned my addr 172.21.15.86:0/3576681712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1762800700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17641027e0 0x7f176419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 -- 172.21.15.86:0/3576681712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17641027e0 msgr2=0x7f176419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17641027e0 0x7f176419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 -- 172.21.15.86:0/3576681712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1764100c50 msgr2=0x7f1764196bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-22T19:09:04.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1764100c50 0x7f1764196bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 -- 172.21.15.86:0/3576681712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17540095f0 con 0x7f1764101e40 2024-03-22T19:09:04.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1761400700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 0x7f1764197110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f175000d950 tx=0x7f175000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:04.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750009940 con 0x7f1764101e40 2024-03-22T19:09:04.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176419ecb0 con 0x7f1764101e40 2024-03-22T19:09:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.285+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1750010460 con 0x7f1764101e40 2024-03-22T19:09:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.290+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000f620 con 0x7f1764101e40 2024-03-22T19:09:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.290+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176419f1c0 con 0x7f1764101e40 2024-03-22T19:09:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.290+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17641080d0 con 0x7f1764101e40 2024-03-22T19:09:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.293+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f17500108d0 con 0x7f1764101e40 2024-03-22T19:09:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.293+0000 7f175aa00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17480a0a90 0x7f17480a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.293+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17500cbaa0 con 0x7f1764101e40 2024-03-22T19:09:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.293+0000 7f1761e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17480a0a90 0x7f17480a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.293+0000 7f1761e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17480a0a90 0x7f17480a2f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17540098e0 tx=0x7f1754000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.298+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1750095870 con 0x7f1764101e40 2024-03-22T19:09:04.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.559+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17640679e0 con 0x7f1764101e40 2024-03-22T19:09:04.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.560+0000 7f175aa00700 1 -- 172.21.15.86:0/3576681712 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8091 (secure 0 0 0) 0x7f1750098e90 con 0x7f1764101e40 2024-03-22T19:09:04.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:04.562 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17480a0a90 msgr2=0x7f17480a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17480a0a90 0x7f17480a2f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17540098e0 tx=0x7f1754000bc0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 msgr2=0x7f1764197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 0x7f1764197110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f175000d950 tx=0x7f175000dc60 comp rx=0 tx=0).stop 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 shutdown_connections 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17480a0a90 0x7f17480a2f50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17641027e0 0x7f176419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1764100c50 0x7f1764196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 --2- 172.21.15.86:0/3576681712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764101e40 0x7f1764197110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.562+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 >> 172.21.15.86:0/3576681712 conn(0x7f17640fc1c0 msgr2=0x7f1764105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.563+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 shutdown_connections 2024-03-22T19:09:04.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:04.563+0000 7f1768e00700 1 -- 172.21.15.86:0/3576681712 wait complete. 2024-03-22T19:09:04.565 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 11 2024-03-22T19:09:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:04 smithi167 ceph-mon[47728]: Deploying daemon mds.a on smithi086 2024-03-22T19:09:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:04 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6838/3022543840,v1:172.21.15.86:6839/3022543840] up:boot 2024-03-22T19:09:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:04 smithi167 ceph-mon[47728]: fsmap 10 up:standby 2024-03-22T19:09:04.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:04 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-22T19:09:04.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:04 smithi086 ceph-mon[38032]: Deploying daemon mds.a on smithi086 2024-03-22T19:09:04.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:04 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6838/3022543840,v1:172.21.15.86:6839/3022543840] up:boot 2024-03-22T19:09:04.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:04 smithi086 ceph-mon[38032]: fsmap 10 up:standby 2024-03-22T19:09:04.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:04 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-22T19:09:04.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:04 smithi131 ceph-mon[78559]: Deploying daemon mds.a on smithi086 2024-03-22T19:09:04.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:04 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6838/3022543840,v1:172.21.15.86:6839/3022543840] up:boot 2024-03-22T19:09:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:04 smithi131 ceph-mon[78559]: fsmap 10 up:standby 2024-03-22T19:09:04.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:04 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-22T19:09:05.259 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:09:05 smithi086 systemd[1]: Starting Ceph mds.a for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:09:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:05 smithi167 ceph-mon[47728]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:05 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3576681712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:05.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:05 smithi131 ceph-mon[78559]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:05 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3576681712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:05 smithi086 ceph-mon[38032]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:06.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:05 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3576681712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:06.009 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:09:05 smithi086 bash[90433]: 7b0ff2dfbc7f2888ba3e8aa32dded910421a75b312acf5c9851cb52ad5947d45 2024-03-22T19:09:06.009 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:09:05 smithi086 systemd[1]: Started Ceph mds.a for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:09:06.009 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:09:05 smithi086 ceph-mds[90471]: -- 172.21.15.86:0/3405981122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf532441a0 con 0x55cf5321d000 2024-03-22T19:09:06.009 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:09:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a[90448]: starting mds.a at 2024-03-22T19:09:06.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 -- 172.21.15.86:0/732362780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 msgr2=0x7fe79c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 --2- 172.21.15.86:0/732362780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c101070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0099c0 tx=0x7fe78c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 -- 172.21.15.86:0/732362780 shutdown_connections 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 --2- 172.21.15.86:0/732362780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe79c1027e0 0x7fe79c109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 --2- 172.21.15.86:0/732362780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c101070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 --2- 172.21.15.86:0/732362780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 0x7fe79c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 -- 172.21.15.86:0/732362780 >> 172.21.15.86:0/732362780 conn(0x7fe79c0fc1c0 msgr2=0x7fe79c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 -- 172.21.15.86:0/732362780 shutdown_connections 2024-03-22T19:09:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.789+0000 7fe7a1400700 1 -- 172.21.15.86:0/732362780 wait complete. 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.790+0000 7fe7a1400700 1 Processor -- start 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.790+0000 7fe7a1400700 1 -- start start 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe7a1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe7a1400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 0x7fe79c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe7a1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe79c1027e0 0x7fe79c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe7a1400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1a0970 con 0x7fe79c1027e0 2024-03-22T19:09:06.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe7a1400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1a0ab0 con 0x7fe79c100c50 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe7a1400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1a0c40 con 0x7fe79c101e40 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe79c1027e0 0x7fe79c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:54210/0 (socket says 172.21.15.86:54210) 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 -- 172.21.15.86:0/1266897070 learned_addr learned my addr 172.21.15.86:0/1266897070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe793e00700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 0x7fe79c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 -- 172.21.15.86:0/1266897070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 msgr2=0x7fe79c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 0x7fe79c1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 -- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe79c1027e0 msgr2=0x7fe79c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe79c1027e0 0x7fe79c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe79a800700 1 -- 172.21.15.86:0/1266897070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009c40 con 0x7fe79c100c50 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.791+0000 7fe793e00700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 0x7fe79c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.792+0000 7fe79a800700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c198e90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.792+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c01d070 con 0x7fe79c100c50 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.792+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c0095f0 con 0x7fe79c100c50 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.792+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe78c0054e0 con 0x7fe79c100c50 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.792+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c004b30 con 0x7fe79c100c50 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.792+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c1a1150 con 0x7fe79c100c50 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.793+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c0679e0 con 0x7fe79c100c50 2024-03-22T19:09:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.795+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe78c00fbb0 con 0x7fe79c100c50 2024-03-22T19:09:06.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.796+0000 7fe793400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7800a0a90 0x7fe7800a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:06.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.796+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe78c0cc8b0 con 0x7fe79c100c50 2024-03-22T19:09:06.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.796+0000 7fe793e00700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7800a0a90 0x7fe7800a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:06.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.797+0000 7fe793e00700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7800a0a90 0x7fe7800a2f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe784009cb0 tx=0x7fe784009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:06.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:06.802+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe78c096710 con 0x7fe79c100c50 2024-03-22T19:09:07.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: Deploying daemon mds.b on smithi131 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] up:boot 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: fsmap 11 up:standby 2024-03-22T19:09:07.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:06 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: Deploying daemon mds.b on smithi131 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] up:boot 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: fsmap 11 up:standby 2024-03-22T19:09:07.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:06 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-22T19:09:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.052+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe79c1a13d0 con 0x7fe79c100c50 2024-03-22T19:09:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.053+0000 7fe793400700 1 -- 172.21.15.86:0/1266897070 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8849 (secure 0 0 0) 0x7fe78c083e60 con 0x7fe79c100c50 2024-03-22T19:09:07.055 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:07.055 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2024-03-22T19:09:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7800a0a90 msgr2=0x7fe7800a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7800a0a90 0x7fe7800a2f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe784009cb0 tx=0x7fe784009260 comp rx=0 tx=0).stop 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 msgr2=0x7fe79c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c198e90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c00f740 comp rx=0 tx=0).stop 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 shutdown_connections 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7800a0a90 0x7fe7800a2f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe79c1027e0 0x7fe79c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c100c50 0x7fe79c198e90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 --2- 172.21.15.86:0/1266897070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c101e40 0x7fe79c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 >> 172.21.15.86:0/1266897070 conn(0x7fe79c0fc1c0 msgr2=0x7fe79c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 shutdown_connections 2024-03-22T19:09:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:07.056+0000 7fe7a1400700 1 -- 172.21.15.86:0/1266897070 wait complete. 2024-03-22T19:09:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: Deploying daemon mds.b on smithi131 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] up:boot 2024-03-22T19:09:07.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: fsmap 11 up:standby 2024-03-22T19:09:07.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:06 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-22T19:09:07.721 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:09:07 smithi131 bash[116590]: e0e448232279b40a8c06bd0cc3e3c6b6de4808a5efaab728ff103c3d7e891013 2024-03-22T19:09:07.721 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:09:07 smithi131 systemd[1]: Started Ceph mds.b for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:09:08.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:07 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1266897070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:07 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:07 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1266897070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:08.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:07 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:07 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:07 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:07 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:07 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:07 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1266897070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:08.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:07 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:07 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:07 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:07 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:08.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:07 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:09.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:08 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:09:09.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:08 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] up:boot 2024-03-22T19:09:09.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:08 smithi086 ceph-mon[38032]: fsmap 12 up:standby 2024-03-22T19:09:09.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:08 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-22T19:09:09.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:08 smithi086 ceph-mon[38032]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:09.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:09:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:08 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:09:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:08 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] up:boot 2024-03-22T19:09:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:08 smithi167 ceph-mon[47728]: fsmap 12 up:standby 2024-03-22T19:09:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:08 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-22T19:09:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:08 smithi167 ceph-mon[47728]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:09.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:08 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:09:09.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:08 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] up:boot 2024-03-22T19:09:09.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:08 smithi131 ceph-mon[78559]: fsmap 12 up:standby 2024-03-22T19:09:09.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:08 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-22T19:09:09.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:08 smithi131 ceph-mon[78559]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 -- 172.21.15.86:0/2714255223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c073810 msgr2=0x7f6e3c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2714255223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c073810 0x7f6e3c073c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009990 tx=0x7f6e28009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 -- 172.21.15.86:0/2714255223 shutdown_connections 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2714255223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c10d220 0x7f6e3c10f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2714255223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2714255223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c073810 0x7f6e3c073c90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 -- 172.21.15.86:0/2714255223 >> 172.21.15.86:0/2714255223 conn(0x7f6e3c0fc1c0 msgr2=0x7f6e3c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 -- 172.21.15.86:0/2714255223 shutdown_connections 2024-03-22T19:09:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.289+0000 7f6e42a00700 1 -- 172.21.15.86:0/2714255223 wait complete. 2024-03-22T19:09:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 Processor -- start 2024-03-22T19:09:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 -- start start 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c073810 0x7f6e3c19d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c10d220 0x7f6e3c1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c1a4d70 con 0x7f6e3c10d220 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c1a4eb0 con 0x7f6e3c074df0 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.291+0000 7f6e42a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c1a5040 con 0x7f6e3c073810 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c19d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:41802/0 (socket says 172.21.15.86:41802) 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 -- 172.21.15.86:0/2158713932 learned_addr learned my addr 172.21.15.86:0/2158713932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e40a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c10d220 0x7f6e3c1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3be00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c073810 0x7f6e3c19d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 -- 172.21.15.86:0/2158713932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c073810 msgr2=0x7f6e3c19d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c073810 0x7f6e3c19d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 -- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c10d220 msgr2=0x7f6e3c1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c10d220 0x7f6e3c1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 -- 172.21.15.86:0/2158713932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280095f0 con 0x7f6e3c074df0 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e40a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c10d220 0x7f6e3c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e3b400700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c19d780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e28000c00 tx=0x7f6e28005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.292+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801d070 con 0x7f6e3c074df0 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.293+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c1a52c0 con 0x7f6e3c074df0 2024-03-22T19:09:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.293+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e28021410 con 0x7f6e3c074df0 2024-03-22T19:09:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.293+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28017650 con 0x7f6e3c074df0 2024-03-22T19:09:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.293+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c1a56e0 con 0x7f6e3c074df0 2024-03-22T19:09:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.294+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c103480 con 0x7f6e3c074df0 2024-03-22T19:09:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.295+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6e28021580 con 0x7f6e3c074df0 2024-03-22T19:09:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.298+0000 7f6e38c00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e200a0a90 0x7f6e200a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.298+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e280cd150 con 0x7f6e3c074df0 2024-03-22T19:09:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.298+0000 7f6e3be00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e200a0a90 0x7f6e200a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.298+0000 7f6e3be00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e200a0a90 0x7f6e200a2f50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009c60 tx=0x7f6e2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:09.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.301+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6e28096fb0 con 0x7f6e3c074df0 2024-03-22T19:09:09.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.572+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e3c0679e0 con 0x7f6e3c074df0 2024-03-22T19:09:09.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.573+0000 7f6e38c00700 1 -- 172.21.15.86:0/2158713932 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9610 (secure 0 0 0) 0x7f6e28021860 con 0x7f6e3c074df0 2024-03-22T19:09:09.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:09.575 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-03-22T19:09:09.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e200a0a90 msgr2=0x7f6e200a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e200a0a90 0x7f6e200a2f50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009c60 tx=0x7f6e2c009260 comp rx=0 tx=0).stop 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 msgr2=0x7f6e3c19d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c19d780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e28000c00 tx=0x7f6e28005740 comp rx=0 tx=0).stop 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 shutdown_connections 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e200a0a90 0x7f6e200a2f50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e3c10d220 0x7f6e3c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c073810 0x7f6e3c19d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 --2- 172.21.15.86:0/2158713932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e3c074df0 0x7f6e3c19d780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 >> 172.21.15.86:0/2158713932 conn(0x7f6e3c0fc1c0 msgr2=0x7f6e3c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 shutdown_connections 2024-03-22T19:09:09.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.576+0000 7f6e42a00700 1 -- 172.21.15.86:0/2158713932 wait complete. 2024-03-22T19:09:09.579 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-03-22T19:09:09.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 -- 172.21.15.86:0/607157017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 msgr2=0x7f5844101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 --2- 172.21.15.86:0/607157017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844101070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5834009990 tx=0x7f5834009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 -- 172.21.15.86:0/607157017 shutdown_connections 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 --2- 172.21.15.86:0/607157017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58441027e0 0x7f5844109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 --2- 172.21.15.86:0/607157017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844101070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 --2- 172.21.15.86:0/607157017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5844101e40 0x7f58441022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 -- 172.21.15.86:0/607157017 >> 172.21.15.86:0/607157017 conn(0x7f58440fc1c0 msgr2=0x7f58440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 -- 172.21.15.86:0/607157017 shutdown_connections 2024-03-22T19:09:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.796+0000 7f5849800700 1 -- 172.21.15.86:0/607157017 wait complete. 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.797+0000 7f5849800700 1 Processor -- start 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.797+0000 7f5849800700 1 -- start start 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5849800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844106b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5849800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5844101e40 0x7f58441070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5849800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58441027e0 0x7f5844103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5849800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844103ea0 con 0x7f58441027e0 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5849800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844104010 con 0x7f5844101e40 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5849800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58441041d0 con 0x7f5844100c50 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:38962/0 (socket says 172.21.15.86:38962) 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 -- 172.21.15.86:0/1926737471 learned_addr learned my addr 172.21.15.86:0/1926737471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842000700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5844101e40 0x7f58441070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5843400700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58441027e0 0x7f5844103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 -- 172.21.15.86:0/1926737471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5844101e40 msgr2=0x7f58441070b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5844101e40 0x7f58441070b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 -- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58441027e0 msgr2=0x7f5844103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.798+0000 7f5842a00700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58441027e0 0x7f5844103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f5842a00700 1 -- 172.21.15.86:0/1926737471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c009c40 con 0x7f5844100c50 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f5842000700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5844101e40 0x7f58441070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f5842a00700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844106b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5834000c00 tx=0x7f5834009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583401d070 con 0x7f5844100c50 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5834021410 con 0x7f5844100c50 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58340095f0 con 0x7f5844100c50 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834017650 con 0x7f5844100c50 2024-03-22T19:09:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.799+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58441a5580 con 0x7f5844100c50 2024-03-22T19:09:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.800+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58440679e0 con 0x7f5844100c50 2024-03-22T19:09:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.802+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f5834021580 con 0x7f5844100c50 2024-03-22T19:09:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.803+0000 7f583b400700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f58280a0b10 0x7f58280a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.803+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f58340cce00 con 0x7f5844100c50 2024-03-22T19:09:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.804+0000 7f5842000700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f58280a0b10 0x7f58280a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.804+0000 7f5842000700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f58280a0b10 0x7f58280a2fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5830011490 tx=0x7f5830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:09.807+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5834096c60 con 0x7f5844100c50 2024-03-22T19:09:10.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2158713932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:10.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:09 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:09:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.046+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58441a5890 con 0x7f5844100c50 2024-03-22T19:09:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.048+0000 7f583b400700 1 -- 172.21.15.86:0/1926737471 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f583409a280 con 0x7f5844100c50 2024-03-22T19:09:10.050 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:10.050 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:08:35.803767+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:09:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.050+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f58280a0b10 msgr2=0x7f58280a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.050+0000 7f5849800700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f58280a0b10 0x7f58280a2fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5830011490 tx=0x7f5830009260 comp rx=0 tx=0).stop 2024-03-22T19:09:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 msgr2=0x7f5844106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844106b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5834000c00 tx=0x7f5834009580 comp rx=0 tx=0).stop 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 shutdown_connections 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f58280a0b10 0x7f58280a2fd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58441027e0 0x7f5844103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5844100c50 0x7f5844106b70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 --2- 172.21.15.86:0/1926737471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5844101e40 0x7f58441070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 >> 172.21.15.86:0/1926737471 conn(0x7f58440fc1c0 msgr2=0x7f58441076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 shutdown_connections 2024-03-22T19:09:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.051+0000 7f5849800700 1 -- 172.21.15.86:0/1926737471 wait complete. 2024-03-22T19:09:10.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:09:10.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2158713932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:10.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:09 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:09:10.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 -- 172.21.15.86:0/3468529690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4100d50 msgr2=0x7efca4101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 --2- 172.21.15.86:0/3468529690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4100d50 0x7efca4101170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc940099c0 tx=0x7efc94009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 -- 172.21.15.86:0/3468529690 shutdown_connections 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 --2- 172.21.15.86:0/3468529690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca41028e0 0x7efca4109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 --2- 172.21.15.86:0/3468529690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4100d50 0x7efca4101170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 --2- 172.21.15.86:0/3468529690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4101f40 0x7efca41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 -- 172.21.15.86:0/3468529690 >> 172.21.15.86:0/3468529690 conn(0x7efca40fc2c0 msgr2=0x7efca40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.290+0000 7efca8a00700 1 -- 172.21.15.86:0/3468529690 shutdown_connections 2024-03-22T19:09:10.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.291+0000 7efca8a00700 1 -- 172.21.15.86:0/3468529690 wait complete. 2024-03-22T19:09:10.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.292+0000 7efca8a00700 1 Processor -- start 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.292+0000 7efca8a00700 1 -- start start 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca8a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 0x7efca4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4101f40 0x7efca40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca8a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca41028e0 0x7efca40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4073e80 con 0x7efca4101f40 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca8a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4073ff0 con 0x7efca4100d50 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca8a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a0d60 con 0x7efca41028e0 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 0x7efca4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4101f40 0x7efca40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 0x7efca4074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:41838/0 (socket says 172.21.15.86:41838) 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 -- 172.21.15.86:0/1478913476 learned_addr learned my addr 172.21.15.86:0/1478913476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca2800700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca41028e0 0x7efca40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 -- 172.21.15.86:0/1478913476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca41028e0 msgr2=0x7efca40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca41028e0 0x7efca40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 -- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4101f40 msgr2=0x7efca40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4101f40 0x7efca40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 -- 172.21.15.86:0/1478913476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940095f0 con 0x7efca4100d50 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.293+0000 7efca1e00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 0x7efca4074b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9400b580 tx=0x7efc94005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.294+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9401d070 con 0x7efca4100d50 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.294+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc94005630 con 0x7efca4100d50 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.294+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca41a0f90 con 0x7efca4100d50 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.294+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9400fba0 con 0x7efca4100d50 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.294+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca41a1300 con 0x7efca4100d50 2024-03-22T19:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.295+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca4108240 con 0x7efca4100d50 2024-03-22T19:09:10.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.297+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7efc940037b0 con 0x7efca4100d50 2024-03-22T19:09:10.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.298+0000 7efc9aa00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efc880a0b10 0x7efc880a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.298+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efc940cc940 con 0x7efca4100d50 2024-03-22T19:09:10.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.298+0000 7efca1400700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efc880a0b10 0x7efc880a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.299+0000 7efca1400700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efc880a0b10 0x7efc880a2fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc90009c60 tx=0x7efc90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.302+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc940967a0 con 0x7efca4100d50 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2158713932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:10.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:09 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:09:10.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.526+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efca402cee0 con 0x7efca4100d50 2024-03-22T19:09:10.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.527+0000 7efc9aa00700 1 -- 172.21.15.86:0/1478913476 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc94099dc0 con 0x7efca4100d50 2024-03-22T19:09:10.528 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:09:10.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efc880a0b10 msgr2=0x7efc880a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efc880a0b10 0x7efc880a2fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc90009c60 tx=0x7efc90009260 comp rx=0 tx=0).stop 2024-03-22T19:09:10.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 msgr2=0x7efca4074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 0x7efca4074b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9400b580 tx=0x7efc94005280 comp rx=0 tx=0).stop 2024-03-22T19:09:10.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 shutdown_connections 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efc880a0b10 0x7efc880a2fd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca41028e0 0x7efca40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efca4100d50 0x7efca4074b10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 --2- 172.21.15.86:0/1478913476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca4101f40 0x7efca40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 >> 172.21.15.86:0/1478913476 conn(0x7efca40fc2c0 msgr2=0x7efca4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 shutdown_connections 2024-03-22T19:09:10.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.529+0000 7efca8a00700 1 -- 172.21.15.86:0/1478913476 wait complete. 2024-03-22T19:09:10.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T19:09:10.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 -- 172.21.15.86:0/1696916783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2760103fe0 msgr2=0x7f27601063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 --2- 172.21.15.86:0/1696916783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2760103fe0 0x7f27601063d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f274c009990 tx=0x7f274c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 -- 172.21.15.86:0/1696916783 shutdown_connections 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 --2- 172.21.15.86:0/1696916783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 0x7f2760108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 --2- 172.21.15.86:0/1696916783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27601016b0 0x7f2760103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 --2- 172.21.15.86:0/1696916783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2760103fe0 0x7f27601063d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 -- 172.21.15.86:0/1696916783 >> 172.21.15.86:0/1696916783 conn(0x7f27600fb0a0 msgr2=0x7f27600fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 -- 172.21.15.86:0/1696916783 shutdown_connections 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.754+0000 7f2766a00700 1 -- 172.21.15.86:0/1696916783 wait complete. 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.755+0000 7f2766a00700 1 Processor -- start 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.755+0000 7f2766a00700 1 -- start start 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2766a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27601016b0 0x7f2760198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2766a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 0x7f27601992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2766a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 0x7f27601a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2766a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a08e0 con 0x7f27601016b0 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2766a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a0a20 con 0x7f2760103fe0 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2766a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a0bb0 con 0x7f2760106910 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 0x7f27601992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27601016b0 0x7f2760198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2764a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 0x7f27601a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 0x7f27601992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:41844/0 (socket says 172.21.15.86:41844) 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 -- 172.21.15.86:0/2625931095 learned_addr learned my addr 172.21.15.86:0/2625931095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 -- 172.21.15.86:0/2625931095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 msgr2=0x7f27601a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 0x7f27601a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 -- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27601016b0 msgr2=0x7f2760198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27601016b0 0x7f2760198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275f400700 1 -- 172.21.15.86:0/2625931095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c0095f0 con 0x7f2760103fe0 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f2764a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 0x7f27601a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.756+0000 7f275fe00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27601016b0 0x7f2760198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.757+0000 7f275f400700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 0x7f27601992f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f274c005910 tx=0x7f274c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.757+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c01d070 con 0x7f2760103fe0 2024-03-22T19:09:10.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.757+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f274c021410 con 0x7f2760103fe0 2024-03-22T19:09:10.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.757+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27601a0e30 con 0x7f2760103fe0 2024-03-22T19:09:10.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.757+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c005720 con 0x7f2760103fe0 2024-03-22T19:09:10.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.757+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27601a11f0 con 0x7f2760103fe0 2024-03-22T19:09:10.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.758+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27601084e0 con 0x7f2760103fe0 2024-03-22T19:09:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.760+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f274c021580 con 0x7f2760103fe0 2024-03-22T19:09:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.761+0000 7f275cc00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f27440a0a90 0x7f27440a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.761+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f274c0cd900 con 0x7f2760103fe0 2024-03-22T19:09:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.762+0000 7f275fe00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f27440a0a90 0x7f27440a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.762+0000 7f275fe00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f27440a0a90 0x7f27440a2f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2750011490 tx=0x7f2750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:10.764+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f274c097760 con 0x7f2760103fe0 2024-03-22T19:09:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.052+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f276002ccb0 con 0x7f2760103fe0 2024-03-22T19:09:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.053+0000 7f275cc00700 1 -- 172.21.15.86:0/2625931095 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7f274c0a1030 con 0x7f2760103fe0 2024-03-22T19:09:11.054 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:11.054 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T19:09:11.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:09:11.054 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T19:09:11.055 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T19:09:11.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:09:11.055 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f27440a0a90 msgr2=0x7f27440a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f27440a0a90 0x7f27440a2f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2750011490 tx=0x7f2750009260 comp rx=0 tx=0).stop 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 msgr2=0x7f27601992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 0x7f27601992f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f274c005910 tx=0x7f274c009580 comp rx=0 tx=0).stop 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 shutdown_connections 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f27440a0a90 0x7f27440a2f50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2760106910 0x7f27601a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27601016b0 0x7f2760198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 --2- 172.21.15.86:0/2625931095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2760103fe0 0x7f27601992f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 >> 172.21.15.86:0/2625931095 conn(0x7f27600fb0a0 msgr2=0x7f2760100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 shutdown_connections 2024-03-22T19:09:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.055+0000 7f2766a00700 1 -- 172.21.15.86:0/2625931095 wait complete. 2024-03-22T19:09:11.076 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T19:09:11.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1926737471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1478913476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:10 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1926737471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:11.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:11.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1478913476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:09:11.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:10 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 -- 172.21.15.86:0/1108432515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 msgr2=0x7f0fb0101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 --2- 172.21.15.86:0/1108432515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0101170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009990 tx=0x7f0fa0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 -- 172.21.15.86:0/1108432515 shutdown_connections 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 --2- 172.21.15.86:0/1108432515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb01028e0 0x7f0fb0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 --2- 172.21.15.86:0/1108432515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0101170 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 --2- 172.21.15.86:0/1108432515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fb0101f40 0x7f0fb01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 -- 172.21.15.86:0/1108432515 >> 172.21.15.86:0/1108432515 conn(0x7f0fb00fc2c0 msgr2=0x7f0fb00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:11.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 -- 172.21.15.86:0/1108432515 shutdown_connections 2024-03-22T19:09:11.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.294+0000 7f0fb6000700 1 -- 172.21.15.86:0/1108432515 wait complete. 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.295+0000 7f0fb6000700 1 Processor -- start 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 -- start start 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb0101f40 0x7f0fb01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fb01028e0 0x7f0fb01a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb01a0a90 con 0x7f0fb0101f40 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb01a0bd0 con 0x7f0fb01028e0 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fb6000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb01a0d60 con 0x7f0fb0100d50 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0faf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0faf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:39008/0 (socket says 172.21.15.86:39008) 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0faea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb0101f40 0x7f0fb01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0faf400700 1 -- 172.21.15.86:0/202709730 learned_addr learned my addr 172.21.15.86:0/202709730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.296+0000 7f0fafe00700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fb01028e0 0x7f0fb01a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0faf400700 1 -- 172.21.15.86:0/202709730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fb01028e0 msgr2=0x7f0fb01a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0faf400700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fb01028e0 0x7f0fb01a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0faf400700 1 -- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb0101f40 msgr2=0x7f0fb01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0faf400700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb0101f40 0x7f0fb01994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0faf400700 1 -- 172.21.15.86:0/202709730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00095f0 con 0x7f0fb0100d50 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0faf400700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0198f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000c00 tx=0x7f0fa0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa001d070 con 0x7f0fb0100d50 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa0021410 con 0x7f0fb0100d50 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb01a0fe0 con 0x7f0fb0100d50 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0017650 con 0x7f0fb0100d50 2024-03-22T19:09:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.297+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb01a1350 con 0x7f0fb0100d50 2024-03-22T19:09:11.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.298+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb0067ac0 con 0x7f0fb0100d50 2024-03-22T19:09:11.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.300+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0fa0021580 con 0x7f0fb0100d50 2024-03-22T19:09:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.301+0000 7f0fa7e00700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f940a0a90 0x7f0f940a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.301+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0fa00cd150 con 0x7f0fb0100d50 2024-03-22T19:09:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.302+0000 7f0faea00700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f940a0a90 0x7f0f940a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.302+0000 7f0faea00700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f940a0a90 0x7f0f940a2f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005950 tx=0x7f0f9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.305+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0fa0096fb0 con 0x7f0fb0100d50 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1926737471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1478913476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:10 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:11.543+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0fb01a1540 con 0x7f0fb0100d50 2024-03-22T19:09:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.123+0000 7f0fa7e00700 1 -- 172.21.15.86:0/202709730 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0fa0021860 con 0x7f0fb0100d50 2024-03-22T19:09:12.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f940a0a90 msgr2=0x7f0f940a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f940a0a90 0x7f0f940a2f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005950 tx=0x7f0f9c00b410 comp rx=0 tx=0).stop 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 msgr2=0x7f0fb0198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0198f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000c00 tx=0x7f0fa0005740 comp rx=0 tx=0).stop 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 shutdown_connections 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f940a0a90 0x7f0f940a2f50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fb01028e0 0x7f0fb01a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fb0100d50 0x7f0fb0198f60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 --2- 172.21.15.86:0/202709730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb0101f40 0x7f0fb01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 >> 172.21.15.86:0/202709730 conn(0x7f0fb00fc2c0 msgr2=0x7f0fb0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 shutdown_connections 2024-03-22T19:09:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.126+0000 7f0fb6000700 1 -- 172.21.15.86:0/202709730 wait complete. 2024-03-22T19:09:12.129 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T19:09:12.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T19:09:12.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:12 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:12.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2625931095' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 -- 172.21.15.86:0/4291730033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d00ff620 msgr2=0x7fe3d00ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 --2- 172.21.15.86:0/4291730033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d00ff620 0x7fe3d00ffa40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009990 tx=0x7fe3c0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 -- 172.21.15.86:0/4291730033 shutdown_connections 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 --2- 172.21.15.86:0/4291730033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d0100940 0x7fe3d0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 --2- 172.21.15.86:0/4291730033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d00ff620 0x7fe3d00ffa40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 --2- 172.21.15.86:0/4291730033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 0x7fe3d0100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 -- 172.21.15.86:0/4291730033 >> 172.21.15.86:0/4291730033 conn(0x7fe3d00fb1a0 msgr2=0x7fe3d00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 -- 172.21.15.86:0/4291730033 shutdown_connections 2024-03-22T19:09:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.346+0000 7fe3d5200700 1 -- 172.21.15.86:0/4291730033 wait complete. 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.347+0000 7fe3d5200700 1 Processor -- start 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.347+0000 7fe3d5200700 1 -- start start 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3d5200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d00ff620 0x7fe3d0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3d5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 0x7fe3d01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3d5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 0x7fe3d01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3d5200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01a09c0 con 0x7fe3d0100940 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3d5200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01a0b00 con 0x7fe3d00fff80 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3d5200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01a0c90 con 0x7fe3d00ff620 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 0x7fe3d01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 0x7fe3d01a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59706/0 (socket says 172.21.15.86:59706) 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 -- 172.21.15.86:0/722938122 learned_addr learned my addr 172.21.15.86:0/722938122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3cde00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d00ff620 0x7fe3d0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3c7e00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 0x7fe3d01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 -- 172.21.15.86:0/722938122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d00ff620 msgr2=0x7fe3d0198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d00ff620 0x7fe3d0198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 -- 172.21.15.86:0/722938122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 msgr2=0x7fe3d01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 0x7fe3d01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3ce800700 1 -- 172.21.15.86:0/722938122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00095f0 con 0x7fe3d0100940 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3cde00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d00ff620 0x7fe3d0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.348+0000 7fe3c7e00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 0x7fe3d01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.349+0000 7fe3ce800700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 0x7fe3d01a03f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ef30 tx=0x7fe3c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.349+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8018070 con 0x7fe3d0100940 2024-03-22T19:09:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.349+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c800f040 con 0x7fe3d0100940 2024-03-22T19:09:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.349+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d01a0f10 con 0x7fe3d0100940 2024-03-22T19:09:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.349+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800cd00 con 0x7fe3d0100940 2024-03-22T19:09:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.349+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d01a13e0 con 0x7fe3d0100940 2024-03-22T19:09:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.350+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d010a6a0 con 0x7fe3d0100940 2024-03-22T19:09:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.352+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe3c8003a00 con 0x7fe3d0100940 2024-03-22T19:09:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.353+0000 7fe3c6a00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3b40a0b10 0x7fe3b40a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.353+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fe3c80cc1c0 con 0x7fe3d0100940 2024-03-22T19:09:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.354+0000 7fe3cde00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3b40a0b10 0x7fe3b40a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.354+0000 7fe3cde00700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3b40a0b10 0x7fe3b40a2fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000b2e0 tx=0x7fe3c000b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.356+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3c8095eb0 con 0x7fe3d0100940 2024-03-22T19:09:12.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:12 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:12.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2625931095' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-22T19:09:12.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:12 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:12.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2625931095' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-22T19:09:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:12.566+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe3d002d130 con 0x7fe3d0100940 2024-03-22T19:09:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.122+0000 7fe3c6a00700 1 -- 172.21.15.86:0/722938122 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe3d002d130 con 0x7fe3d0100940 2024-03-22T19:09:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.126+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3b40a0b10 msgr2=0x7fe3b40a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.126+0000 7fe3d5200700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3b40a0b10 0x7fe3b40a2fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000b2e0 tx=0x7fe3c000b6f0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.126+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 msgr2=0x7fe3d01a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.126+0000 7fe3d5200700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 0x7fe3d01a03f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ef30 tx=0x7fe3c800c3a0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 shutdown_connections 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3b40a0b10 0x7fe3b40a2fd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d0100940 0x7fe3d01a03f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d00ff620 0x7fe3d0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 --2- 172.21.15.86:0/722938122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00fff80 0x7fe3d01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 >> 172.21.15.86:0/722938122 conn(0x7fe3d00fb1a0 msgr2=0x7fe3d00fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.127+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 shutdown_connections 2024-03-22T19:09:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.128+0000 7fe3d5200700 1 -- 172.21.15.86:0/722938122 wait complete. 2024-03-22T19:09:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T19:09:13.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T19:09:13.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/202709730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-22T19:09:13.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-22T19:09:13.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:13.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:09:13.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-22T19:09:13.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:13 smithi167 ceph-mon[47728]: osdmap e77: 12 total, 12 up, 12 in 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 -- 172.21.15.86:0/4231453791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990101f30 msgr2=0x7f7990102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 --2- 172.21.15.86:0/4231453791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990101f30 0x7f7990102390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f797c0099c0 tx=0x7f797c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 -- 172.21.15.86:0/4231453791 shutdown_connections 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 --2- 172.21.15.86:0/4231453791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79901028d0 0x7f7990109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 --2- 172.21.15.86:0/4231453791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990100d40 0x7f7990101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 --2- 172.21.15.86:0/4231453791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990101f30 0x7f7990102390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 -- 172.21.15.86:0/4231453791 >> 172.21.15.86:0/4231453791 conn(0x7f79900fc2b0 msgr2=0x7f79900fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 -- 172.21.15.86:0/4231453791 shutdown_connections 2024-03-22T19:09:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.355+0000 7f7996800700 1 -- 172.21.15.86:0/4231453791 wait complete. 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.356+0000 7f7996800700 1 Processor -- start 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.356+0000 7f7996800700 1 -- start start 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.356+0000 7f7996800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 0x7f7990198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f7996800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7990101f30 0x7f7990199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f7996800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79901028d0 0x7f79901a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f7996800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901a0a10 con 0x7f7990100d40 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f7996800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901a0b50 con 0x7f79901028d0 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f7996800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901a0ce0 con 0x7f7990101f30 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 0x7f7990198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 0x7f7990198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59736/0 (socket says 172.21.15.86:59736) 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 -- 172.21.15.86:0/74487465 learned_addr learned my addr 172.21.15.86:0/74487465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798fe00700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79901028d0 0x7f79901a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798ea00700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7990101f30 0x7f7990199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 -- 172.21.15.86:0/74487465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7990101f30 msgr2=0x7f7990199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7990101f30 0x7f7990199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 -- 172.21.15.86:0/74487465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79901028d0 msgr2=0x7f79901a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79901028d0 0x7f79901a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 -- 172.21.15.86:0/74487465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0095f0 con 0x7f7990100d40 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798fe00700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79901028d0 0x7f79901a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.357+0000 7f798f400700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 0x7f7990198ee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f798000e970 tx=0x7f798000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.358+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980009790 con 0x7f7990100d40 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.358+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79800044e0 con 0x7f7990100d40 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.358+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79901a0fc0 con 0x7f7990100d40 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.358+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980003e30 con 0x7f7990100d40 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.358+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79901a13f0 con 0x7f7990100d40 2024-03-22T19:09:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.359+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7990067ac0 con 0x7f7990100d40 2024-03-22T19:09:13.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.361+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f7980003f90 con 0x7f7990100d40 2024-03-22T19:09:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.362+0000 7f7987e00700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79740a0b10 0x7f79740a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.362+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f7980014070 con 0x7f7990100d40 2024-03-22T19:09:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.362+0000 7f798ea00700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79740a0b10 0x7f79740a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.363+0000 7f798ea00700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79740a0b10 0x7f79740a2fd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f797c00b580 tx=0x7f797c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.365+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7980095e00 con 0x7f7990100d40 2024-03-22T19:09:13.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/202709730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-22T19:09:13.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-22T19:09:13.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:09:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-22T19:09:13.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:13 smithi131 ceph-mon[78559]: osdmap e77: 12 total, 12 up, 12 in 2024-03-22T19:09:13.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/202709730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-22T19:09:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:13 smithi086 ceph-mon[38032]: osdmap e77: 12 total, 12 up, 12 in 2024-03-22T19:09:13.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.616+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f79901a1670 con 0x7f7990100d40 2024-03-22T19:09:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.623+0000 7f7987e00700 1 -- 172.21.15.86:0/74487465 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7980099420 con 0x7f7990100d40 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.628+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79740a0b10 msgr2=0x7f79740a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.628+0000 7f7996800700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79740a0b10 0x7f79740a2fd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f797c00b580 tx=0x7f797c01a040 comp rx=0 tx=0).stop 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.628+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 msgr2=0x7f7990198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.628+0000 7f7996800700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 0x7f7990198ee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f798000e970 tx=0x7f798000ed30 comp rx=0 tx=0).stop 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 shutdown_connections 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79740a0b10 0x7f79740a2fd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79901028d0 0x7f79901a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990100d40 0x7f7990198ee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 --2- 172.21.15.86:0/74487465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7990101f30 0x7f7990199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 >> 172.21.15.86:0/74487465 conn(0x7f79900fc2b0 msgr2=0x7f7990107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.629+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 shutdown_connections 2024-03-22T19:09:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.630+0000 7f7996800700 1 -- 172.21.15.86:0/74487465 wait complete. 2024-03-22T19:09:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-03-22T19:09:13.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:09:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 -- 172.21.15.86:0/1788843145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0100c50 msgr2=0x7f8bb0101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 --2- 172.21.15.86:0/1788843145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0100c50 0x7f8bb0101070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00098f0 tx=0x7f8ba0009c00 comp rx=0 tx=0).stop 2024-03-22T19:09:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 -- 172.21.15.86:0/1788843145 shutdown_connections 2024-03-22T19:09:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 --2- 172.21.15.86:0/1788843145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 0x7f8bb0109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 --2- 172.21.15.86:0/1788843145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0100c50 0x7f8bb0101070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 --2- 172.21.15.86:0/1788843145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0101e40 0x7f8bb01022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 -- 172.21.15.86:0/1788843145 >> 172.21.15.86:0/1788843145 conn(0x7f8bb00fc1c0 msgr2=0x7f8bb00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 -- 172.21.15.86:0/1788843145 shutdown_connections 2024-03-22T19:09:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.854+0000 7f8bb5400700 1 -- 172.21.15.86:0/1788843145 wait complete. 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.855+0000 7f8bb5400700 1 Processor -- start 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.855+0000 7f8bb5400700 1 -- start start 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bb5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 0x7f8bb0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bb5400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0101e40 0x7f8bb0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bb5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 0x7f8bb01a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bb5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb01a0970 con 0x7f8bb01027e0 2024-03-22T19:09:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bb5400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb01a0ab0 con 0x7f8bb0100c50 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bb5400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb01a0c40 con 0x7f8bb0101e40 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bae800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 0x7f8bb0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8baf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 0x7f8bb01a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bae800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 0x7f8bb0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:41948/0 (socket says 172.21.15.86:41948) 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bae800700 1 -- 172.21.15.86:0/2482652010 learned_addr learned my addr 172.21.15.86:0/2482652010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bade00700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0101e40 0x7f8bb0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8baf200700 1 -- 172.21.15.86:0/2482652010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0101e40 msgr2=0x7f8bb0199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8baf200700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0101e40 0x7f8bb0199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8baf200700 1 -- 172.21.15.86:0/2482652010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 msgr2=0x7f8bb0198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8baf200700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 0x7f8bb0198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8baf200700 1 -- 172.21.15.86:0/2482652010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8009d00 con 0x7f8bb01027e0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.856+0000 7f8bae800700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 0x7f8bb0198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.857+0000 7f8baf200700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 0x7f8bb01a03a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009cd0 tx=0x7f8ba800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.857+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8017070 con 0x7f8bb01027e0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.857+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba0009520 con 0x7f8bb01027e0 2024-03-22T19:09:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.857+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ba800f040 con 0x7f8bb01027e0 2024-03-22T19:09:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.857+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800e050 con 0x7f8bb01027e0 2024-03-22T19:09:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.857+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb01a1230 con 0x7f8bb01027e0 2024-03-22T19:09:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.858+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb00679e0 con 0x7f8bb01027e0 2024-03-22T19:09:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.860+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8ba800cd10 con 0x7f8bb01027e0 2024-03-22T19:09:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.861+0000 7f8ba7400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b940a0b10 0x7f8b940a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.861+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ba80cc330 con 0x7f8bb01027e0 2024-03-22T19:09:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.861+0000 7f8bae800700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b940a0b10 0x7f8b940a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.862+0000 7f8bae800700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b940a0b10 0x7f8b940a2fd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009fd0 tx=0x7f8ba0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:13.863+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8ba8095db0 con 0x7f8bb01027e0 2024-03-22T19:09:14.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.103+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bb01a1420 con 0x7f8bb01027e0 2024-03-22T19:09:14.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.104+0000 7f8ba7400700 1 -- 172.21.15.86:0/2482652010 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22132 (secure 0 0 0) 0x7f8ba80993d0 con 0x7f8bb01027e0 2024-03-22T19:09:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:14.106 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:13.618380+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b940a0b10 msgr2=0x7f8b940a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b940a0b10 0x7f8b940a2fd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009fd0 tx=0x7f8ba0019040 comp rx=0 tx=0).stop 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 msgr2=0x7f8bb01a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 0x7f8bb01a03a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009cd0 tx=0x7f8ba800c620 comp rx=0 tx=0).stop 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 shutdown_connections 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b940a0b10 0x7f8b940a2fd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb01027e0 0x7f8bb01a03a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bb0100c50 0x7f8bb0198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 --2- 172.21.15.86:0/2482652010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bb0101e40 0x7f8bb0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 >> 172.21.15.86:0/2482652010 conn(0x7f8bb00fc1c0 msgr2=0x7f8bb0107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.107+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 shutdown_connections 2024-03-22T19:09:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.108+0000 7f8bb5400700 1 -- 172.21.15.86:0/2482652010 wait complete. 2024-03-22T19:09:14.128 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 -- 172.21.15.86:0/367605887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 msgr2=0x7f38a4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 --2- 172.21.15.86:0/367605887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4101070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f38980099c0 tx=0x7f3898009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 -- 172.21.15.86:0/367605887 shutdown_connections 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 --2- 172.21.15.86:0/367605887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a41027e0 0x7f38a4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 --2- 172.21.15.86:0/367605887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4101070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 --2- 172.21.15.86:0/367605887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 0x7f38a41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 -- 172.21.15.86:0/367605887 >> 172.21.15.86:0/367605887 conn(0x7f38a40fc1c0 msgr2=0x7f38a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:14.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 -- 172.21.15.86:0/367605887 shutdown_connections 2024-03-22T19:09:14.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.317+0000 7f38ac600700 1 -- 172.21.15.86:0/367605887 wait complete. 2024-03-22T19:09:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.318+0000 7f38ac600700 1 Processor -- start 2024-03-22T19:09:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.318+0000 7f38ac600700 1 -- start start 2024-03-22T19:09:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38ac600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38ac600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 0x7f38a40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38ac600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a41027e0 0x7f38a40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38ac600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4073e80 con 0x7f38a4100c50 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38ac600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4073ff0 con 0x7f38a4101e40 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38ac600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a0c90 con 0x7f38a41027e0 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59770/0 (socket says 172.21.15.86:59770) 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 -- 172.21.15.86:0/233971496 learned_addr learned my addr 172.21.15.86:0/233971496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9200700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 0x7f38a40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38aa600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a41027e0 0x7f38a40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 -- 172.21.15.86:0/233971496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a41027e0 msgr2=0x7f38a40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a41027e0 0x7f38a40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 -- 172.21.15.86:0/233971496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 msgr2=0x7f38a40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 0x7f38a40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9c00700 1 -- 172.21.15.86:0/233971496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980095f0 con 0x7f38a4100c50 2024-03-22T19:09:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.319+0000 7f38a9200700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 0x7f38a40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.320+0000 7f38a9c00700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4074b10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3898006010 tx=0x7f389800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.320+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389801d070 con 0x7f38a4100c50 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.320+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3898005020 con 0x7f38a4100c50 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.320+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a41a0ec0 con 0x7f38a4100c50 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.320+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898004b10 con 0x7f38a4100c50 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.320+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a41a1230 con 0x7f38a4100c50 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.321+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4108120 con 0x7f38a4100c50 2024-03-22T19:09:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.323+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f3898005190 con 0x7f38a4100c50 2024-03-22T19:09:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.324+0000 7f38a2000700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f388c0a0b10 0x7f388c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.324+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f38980cc910 con 0x7f38a4100c50 2024-03-22T19:09:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.324+0000 7f38a9200700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f388c0a0b10 0x7f388c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.325+0000 7f38a9200700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f388c0a0b10 0x7f388c0a2fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3890011490 tx=0x7f3890009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: pgmap v228: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/722938122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/722938122' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: osdmap e78: 12 total, 12 up, 12 in 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/74487465' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: osdmap e79: 12 total, 12 up, 12 in 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/74487465' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-22T19:09:14.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:14 smithi167 ceph-mon[47728]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-22T19:09:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.328+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3898096420 con 0x7f38a4100c50 2024-03-22T19:09:14.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: pgmap v228: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/722938122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/722938122' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: osdmap e78: 12 total, 12 up, 12 in 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/74487465' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: osdmap e79: 12 total, 12 up, 12 in 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/74487465' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-22T19:09:14.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:14 smithi131 ceph-mon[78559]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-22T19:09:14.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: pgmap v228: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/722938122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/722938122' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: osdmap e78: 12 total, 12 up, 12 in 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/74487465' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: osdmap e79: 12 total, 12 up, 12 in 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/74487465' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-22T19:09:14.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:14 smithi086 ceph-mon[38032]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-22T19:09:14.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.565+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38a40679e0 con 0x7f38a4100c50 2024-03-22T19:09:14.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.566+0000 7f38a2000700 1 -- 172.21.15.86:0/233971496 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22132 (secure 0 0 0) 0x7f3898099a40 con 0x7f38a4100c50 2024-03-22T19:09:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:14.568 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:13.618380+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:09:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f388c0a0b10 msgr2=0x7f388c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f388c0a0b10 0x7f388c0a2fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3890011490 tx=0x7f3890009260 comp rx=0 tx=0).stop 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 msgr2=0x7f38a4074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4074b10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3898006010 tx=0x7f389800f690 comp rx=0 tx=0).stop 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 shutdown_connections 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f388c0a0b10 0x7f388c0a2fd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a41027e0 0x7f38a40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a4100c50 0x7f38a4074b10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 --2- 172.21.15.86:0/233971496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38a4101e40 0x7f38a40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 >> 172.21.15.86:0/233971496 conn(0x7f38a40fc1c0 msgr2=0x7f38a4105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 shutdown_connections 2024-03-22T19:09:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.569+0000 7f38ac600700 1 -- 172.21.15.86:0/233971496 wait complete. 2024-03-22T19:09:14.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 -- 172.21.15.86:0/2806247825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204101f30 msgr2=0x7f5204102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 --2- 172.21.15.86:0/2806247825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204101f30 0x7f5204102390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51f40099c0 tx=0x7f51f4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 -- 172.21.15.86:0/2806247825 shutdown_connections 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 --2- 172.21.15.86:0/2806247825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52041028d0 0x7f5204109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 --2- 172.21.15.86:0/2806247825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5204100d40 0x7f5204101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 --2- 172.21.15.86:0/2806247825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204101f30 0x7f5204102390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.784+0000 7f520c800700 1 -- 172.21.15.86:0/2806247825 >> 172.21.15.86:0/2806247825 conn(0x7f52040fc2b0 msgr2=0x7f52040fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:14.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.785+0000 7f520c800700 1 -- 172.21.15.86:0/2806247825 shutdown_connections 2024-03-22T19:09:14.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.785+0000 7f520c800700 1 -- 172.21.15.86:0/2806247825 wait complete. 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 Processor -- start 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 -- start start 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 0x7f5204074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5204101f30 0x7f52040731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52041028d0 0x7f52040736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5204073e80 con 0x7f5204100d40 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5204073ff0 con 0x7f52041028d0 2024-03-22T19:09:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520c800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041a0d40 con 0x7f5204101f30 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f5209e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 0x7f5204074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f5209e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 0x7f5204074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59780/0 (socket says 172.21.15.86:59780) 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f5209e00700 1 -- 172.21.15.86:0/770342957 learned_addr learned my addr 172.21.15.86:0/770342957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f5209400700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5204101f30 0x7f52040731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.786+0000 7f520a800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52041028d0 0x7f52040736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f5209e00700 1 -- 172.21.15.86:0/770342957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5204101f30 msgr2=0x7f52040731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f5209e00700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5204101f30 0x7f52040731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f5209e00700 1 -- 172.21.15.86:0/770342957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52041028d0 msgr2=0x7f52040736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f5209e00700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52041028d0 0x7f52040736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f5209e00700 1 -- 172.21.15.86:0/770342957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f40095f0 con 0x7f5204100d40 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f5209e00700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 0x7f5204074b10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51f800e970 tx=0x7f51f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8009790 con 0x7f5204100d40 2024-03-22T19:09:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f800c780 con 0x7f5204100d40 2024-03-22T19:09:14.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52041a0fd0 con 0x7f5204100d40 2024-03-22T19:09:14.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8003be0 con 0x7f5204100d40 2024-03-22T19:09:14.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.787+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52041a1400 con 0x7f5204100d40 2024-03-22T19:09:14.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.788+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52041081c0 con 0x7f5204100d40 2024-03-22T19:09:14.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.790+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f51f800cbf0 con 0x7f5204100d40 2024-03-22T19:09:14.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.791+0000 7f51fea00700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51ec0a0b10 0x7f51ec0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:14.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.792+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51f8014070 con 0x7f5204100d40 2024-03-22T19:09:14.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.792+0000 7f5209400700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51ec0a0b10 0x7f51ec0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:14.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.792+0000 7f5209400700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51ec0a0b10 0x7f51ec0a2fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51f4003730 tx=0x7f51f400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:14.795+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f51f8095b00 con 0x7f5204100d40 2024-03-22T19:09:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.051+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f520402cf60 con 0x7f5204100d40 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2482652010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: pgmap v231: 97 pgs: 9 creating+peering, 2 active+clean, 86 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/233971496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] up:active 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: osdmap e80: 12 total, 12 up, 12 in 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:15.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:15 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/770342957' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-22T19:09:15.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2482652010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:15.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: pgmap v231: 97 pgs: 9 creating+peering, 2 active+clean, 86 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/233971496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] up:active 2024-03-22T19:09:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: osdmap e80: 12 total, 12 up, 12 in 2024-03-22T19:09:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:15.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:15 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/770342957' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-22T19:09:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2482652010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: pgmap v231: 97 pgs: 9 creating+peering, 2 active+clean, 86 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:09:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/233971496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] up:active 2024-03-22T19:09:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: osdmap e80: 12 total, 12 up, 12 in 2024-03-22T19:09:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:15 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/770342957' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-22T19:09:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.631+0000 7f51fea00700 1 -- 172.21.15.86:0/770342957 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f51f8099120 con 0x7f5204100d40 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51ec0a0b10 msgr2=0x7f51ec0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51ec0a0b10 0x7f51ec0a2fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51f4003730 tx=0x7f51f400b540 comp rx=0 tx=0).stop 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 msgr2=0x7f5204074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 0x7f5204074b10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51f800e970 tx=0x7f51f800ec80 comp rx=0 tx=0).stop 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 shutdown_connections 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51ec0a0b10 0x7f51ec0a2fd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52041028d0 0x7f52040736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5204100d40 0x7f5204074b10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 --2- 172.21.15.86:0/770342957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5204101f30 0x7f52040731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.633+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 >> 172.21.15.86:0/770342957 conn(0x7f52040fc2b0 msgr2=0x7f5204105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.634+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 shutdown_connections 2024-03-22T19:09:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.634+0000 7f520c800700 1 -- 172.21.15.86:0/770342957 wait complete. 2024-03-22T19:09:15.655 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-22T19:09:15.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-22T19:09:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 -- 172.21.15.86:0/645540190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 msgr2=0x7fddfc1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 --2- 172.21.15.86:0/645540190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc1023a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdde80099c0 tx=0x7fdde8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 -- 172.21.15.86:0/645540190 shutdown_connections 2024-03-22T19:09:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 --2- 172.21.15.86:0/645540190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddfc1028e0 0x7fddfc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 --2- 172.21.15.86:0/645540190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddfc100d50 0x7fddfc101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 --2- 172.21.15.86:0/645540190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc1023a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 -- 172.21.15.86:0/645540190 >> 172.21.15.86:0/645540190 conn(0x7fddfc0fc2c0 msgr2=0x7fddfc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 -- 172.21.15.86:0/645540190 shutdown_connections 2024-03-22T19:09:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.849+0000 7fde03800700 1 -- 172.21.15.86:0/645540190 wait complete. 2024-03-22T19:09:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.850+0000 7fde03800700 1 Processor -- start 2024-03-22T19:09:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.850+0000 7fde03800700 1 -- start start 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde03800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddfc100d50 0x7fddfc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde03800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde03800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddfc1028e0 0x7fddfc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde03800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc073e80 con 0x7fddfc101f40 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde03800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc073ff0 con 0x7fddfc100d50 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde03800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc1a0d50 con 0x7fddfc1028e0 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59798/0 (socket says 172.21.15.86:59798) 2024-03-22T19:09:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 -- 172.21.15.86:0/724033675 learned_addr learned my addr 172.21.15.86:0/724033675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde00e00700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddfc100d50 0x7fddfc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fde01800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddfc1028e0 0x7fddfc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 -- 172.21.15.86:0/724033675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddfc1028e0 msgr2=0x7fddfc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddfc1028e0 0x7fddfc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 -- 172.21.15.86:0/724033675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddfc100d50 msgr2=0x7fddfc074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddfc100d50 0x7fddfc074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.851+0000 7fddfbe00700 1 -- 172.21.15.86:0/724033675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80095f0 con 0x7fddfc101f40 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.852+0000 7fddfbe00700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc0731a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006010 tx=0x7fdde800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.852+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde801d070 con 0x7fddfc101f40 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.852+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde8005030 con 0x7fddfc101f40 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.852+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc1a0f80 con 0x7fddfc101f40 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.852+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8004b10 con 0x7fddfc101f40 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.852+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc1a12f0 con 0x7fddfc101f40 2024-03-22T19:09:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.853+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc1081d0 con 0x7fddfc101f40 2024-03-22T19:09:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.855+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fdde80051a0 con 0x7fddfc101f40 2024-03-22T19:09:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.856+0000 7fddf9600700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdde40a0a90 0x7fdde40a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.856+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdde80ccc70 con 0x7fddfc101f40 2024-03-22T19:09:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.856+0000 7fde00e00700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdde40a0a90 0x7fdde40a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.857+0000 7fde00e00700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdde40a0a90 0x7fdde40a2f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddfc101da0 tx=0x7fddf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:15.860+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdde8096780 con 0x7fddfc101f40 2024-03-22T19:09:16.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:16.117+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fddfc02ccb0 con 0x7fddfc101f40 2024-03-22T19:09:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:16 smithi167 ceph-mon[47728]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T19:09:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:16 smithi167 ceph-mon[47728]: Cluster is now healthy 2024-03-22T19:09:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:16 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/770342957' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-22T19:09:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:16 smithi167 ceph-mon[47728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:16.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:16 smithi131 ceph-mon[78559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T19:09:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:16 smithi131 ceph-mon[78559]: Cluster is now healthy 2024-03-22T19:09:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:16 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/770342957' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-22T19:09:16.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:16 smithi131 ceph-mon[78559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:16 smithi086 ceph-mon[38032]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T19:09:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:16 smithi086 ceph-mon[38032]: Cluster is now healthy 2024-03-22T19:09:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:16 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/770342957' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-22T19:09:16.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:16 smithi086 ceph-mon[38032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.124+0000 7fddf9600700 1 -- 172.21.15.86:0/724033675 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fdde8099da0 con 0x7fddfc101f40 2024-03-22T19:09:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdde40a0a90 msgr2=0x7fdde40a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdde40a0a90 0x7fdde40a2f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddfc101da0 tx=0x7fddf0009260 comp rx=0 tx=0).stop 2024-03-22T19:09:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 msgr2=0x7fddfc0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc0731a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006010 tx=0x7fdde800f690 comp rx=0 tx=0).stop 2024-03-22T19:09:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 shutdown_connections 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdde40a0a90 0x7fdde40a2f50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddfc1028e0 0x7fddfc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddfc100d50 0x7fddfc074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 --2- 172.21.15.86:0/724033675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc101f40 0x7fddfc0731a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 >> 172.21.15.86:0/724033675 conn(0x7fddfc0fc2c0 msgr2=0x7fddfc105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 shutdown_connections 2024-03-22T19:09:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.128+0000 7fde03800700 1 -- 172.21.15.86:0/724033675 wait complete. 2024-03-22T19:09:17.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-22T19:09:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 -- 172.21.15.86:0/2573296691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901028d0 msgr2=0x7f3a90109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 --2- 172.21.15.86:0/2573296691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901028d0 0x7f3a90109180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b0d0 tx=0x7f3a8800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 -- 172.21.15.86:0/2573296691 shutdown_connections 2024-03-22T19:09:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 --2- 172.21.15.86:0/2573296691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901028d0 0x7f3a90109180 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 --2- 172.21.15.86:0/2573296691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a90100d40 0x7f3a90101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 --2- 172.21.15.86:0/2573296691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a90101f30 0x7f3a90102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 -- 172.21.15.86:0/2573296691 >> 172.21.15.86:0/2573296691 conn(0x7f3a900fc2b0 msgr2=0x7f3a900fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 -- 172.21.15.86:0/2573296691 shutdown_connections 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.353+0000 7f3a95200700 1 -- 172.21.15.86:0/2573296691 wait complete. 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.354+0000 7f3a95200700 1 Processor -- start 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.354+0000 7f3a95200700 1 -- start start 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a95200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a90100d40 0x7f3a90198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a95200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 0x7f3a90199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a95200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a901028d0 0x7f3a901a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a95200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901a0b20 con 0x7f3a90101f30 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a95200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901a0c60 con 0x7f3a901028d0 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a95200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901a0df0 con 0x7f3a90100d40 2024-03-22T19:09:17.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 0x7f3a90199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 0x7f3a90199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59820/0 (socket says 172.21.15.86:59820) 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 -- 172.21.15.86:0/1563353815 learned_addr learned my addr 172.21.15.86:0/1563353815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8de00700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a90100d40 0x7f3a90198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8e800700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a901028d0 0x7f3a901a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 -- 172.21.15.86:0/1563353815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a90100d40 msgr2=0x7f3a90198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a90100d40 0x7f3a90198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 -- 172.21.15.86:0/1563353815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a901028d0 msgr2=0x7f3a901a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a901028d0 0x7f3a901a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8d400700 1 -- 172.21.15.86:0/1563353815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009520 con 0x7f3a90101f30 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.355+0000 7f3a8e800700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a901028d0 0x7f3a901a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.356+0000 7f3a8d400700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 0x7f3a90199530 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9a0 tx=0x7f3a7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.356+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c009790 con 0x7f3a90101f30 2024-03-22T19:09:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.356+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7c00cbd0 con 0x7f3a90101f30 2024-03-22T19:09:17.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.356+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a88009d00 con 0x7f3a90101f30 2024-03-22T19:09:17.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.356+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c018680 con 0x7f3a90101f30 2024-03-22T19:09:17.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.356+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a901a14a0 con 0x7f3a90101f30 2024-03-22T19:09:17.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.357+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a90067ac0 con 0x7f3a90101f30 2024-03-22T19:09:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.359+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f3a7c00cd40 con 0x7f3a90101f30 2024-03-22T19:09:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.361+0000 7f3a86a00700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3a740a0b10 0x7f3a740a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.361+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3a7c014070 con 0x7f3a90101f30 2024-03-22T19:09:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.361+0000 7f3a8de00700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3a740a0b10 0x7f3a740a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.362+0000 7f3a8de00700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3a740a0b10 0x7f3a740a2fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a90101d90 tx=0x7f3a80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.365+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a7c0529f0 con 0x7f3a90101f30 2024-03-22T19:09:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:17 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/724033675' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-22T19:09:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:17 smithi131 ceph-mon[78559]: pgmap v233: 97 pgs: 9 creating+peering, 53 active+clean, 35 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2024-03-22T19:09:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:17 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/724033675' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-22T19:09:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:17 smithi086 ceph-mon[38032]: pgmap v233: 97 pgs: 9 creating+peering, 53 active+clean, 35 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2024-03-22T19:09:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:17 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/724033675' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-22T19:09:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:17 smithi167 ceph-mon[47728]: pgmap v233: 97 pgs: 9 creating+peering, 53 active+clean, 35 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2024-03-22T19:09:17.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:17.619+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f3a901a16c0 con 0x7f3a90101f30 2024-03-22T19:09:18.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.132+0000 7f3a86a00700 1 -- 172.21.15.86:0/1563353815 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3a7c095bf0 con 0x7f3a90101f30 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3a740a0b10 msgr2=0x7f3a740a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3a740a0b10 0x7f3a740a2fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a90101d90 tx=0x7f3a80009260 comp rx=0 tx=0).stop 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 msgr2=0x7f3a90199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 0x7f3a90199530 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9a0 tx=0x7f3a7c00ecb0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 shutdown_connections 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3a740a0b10 0x7f3a740a2fd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a901028d0 0x7f3a901a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a90100d40 0x7f3a90198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 --2- 172.21.15.86:0/1563353815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a90101f30 0x7f3a90199530 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 >> 172.21.15.86:0/1563353815 conn(0x7f3a900fc2b0 msgr2=0x7f3a90072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 shutdown_connections 2024-03-22T19:09:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.136+0000 7f3a95200700 1 -- 172.21.15.86:0/1563353815 wait complete. 2024-03-22T19:09:18.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-22T19:09:18.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 -- 172.21.15.86:0/1070010840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec108c00 msgr2=0x7f8bec10b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:18.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/1070010840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec108c00 0x7f8bec10b090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8be400b0a0 tx=0x7f8be400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 -- 172.21.15.86:0/1070010840 shutdown_connections 2024-03-22T19:09:18.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/1070010840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec108c00 0x7f8bec10b090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/1070010840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bec074df0 0x7f8bec073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/1070010840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec073810 0x7f8bec073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 -- 172.21.15.86:0/1070010840 >> 172.21.15.86:0/1070010840 conn(0x7f8bec0fc2b0 msgr2=0x7f8bec0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:18.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 -- 172.21.15.86:0/1070010840 shutdown_connections 2024-03-22T19:09:18.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.355+0000 7f8bf3c00700 1 -- 172.21.15.86:0/1070010840 wait complete. 2024-03-22T19:09:18.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.356+0000 7f8bf3c00700 1 Processor -- start 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.356+0000 7f8bf3c00700 1 -- start start 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf3c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 0x7f8bec199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf3c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec074df0 0x7f8bec199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf3c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bec108c00 0x7f8bec1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf3c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1a0b30 con 0x7f8bec073810 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf3c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1a0c70 con 0x7f8bec108c00 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf3c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1a0e00 con 0x7f8bec074df0 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 0x7f8bec199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 0x7f8bec199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59840/0 (socket says 172.21.15.86:59840) 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 -- 172.21.15.86:0/4293179523 learned_addr learned my addr 172.21.15.86:0/4293179523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bebe00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec074df0 0x7f8bec199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bec108c00 0x7f8bec1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 -- 172.21.15.86:0/4293179523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec074df0 msgr2=0x7f8bec199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec074df0 0x7f8bec199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 -- 172.21.15.86:0/4293179523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bec108c00 msgr2=0x7f8bec1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bec108c00 0x7f8bec1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.357+0000 7f8bf1200700 1 -- 172.21.15.86:0/4293179523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4009d00 con 0x7f8bec073810 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.358+0000 7f8bf1200700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 0x7f8bec199000 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8be000e9a0 tx=0x7f8be000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.358+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0009790 con 0x7f8bec073810 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.358+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8be000cbd0 con 0x7f8bec073810 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.358+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0018680 con 0x7f8bec073810 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.358+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec1a10e0 con 0x7f8bec073810 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.358+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec1a15a0 con 0x7f8bec073810 2024-03-22T19:09:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.359+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec1079c0 con 0x7f8bec073810 2024-03-22T19:09:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.361+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8be000cd40 con 0x7f8bec073810 2024-03-22T19:09:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.363+0000 7f8be9600700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8bd40a0b10 0x7f8bd40a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.363+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8be0014070 con 0x7f8bec073810 2024-03-22T19:09:18.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.363+0000 7f8bebe00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8bd40a0b10 0x7f8bd40a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:18.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.364+0000 7f8bebe00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8bd40a0b10 0x7f8bd40a2fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bec074b10 tx=0x7f8bd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:18.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.367+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8be00529f0 con 0x7f8bec073810 2024-03-22T19:09:18.471 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: 2024-03-22T19:09:18.123+0000 7f4cb7fe8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/724033675' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-22T19:09:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:18 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1563353815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-22T19:09:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T19:09:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/724033675' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-22T19:09:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T19:09:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-22T19:09:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-22T19:09:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:18 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1563353815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/724033675' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-22T19:09:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:18 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1563353815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-22T19:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:18.621+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f8bec067ac0 con 0x7f8bec073810 2024-03-22T19:09:19.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.141+0000 7f8be9600700 1 -- 172.21.15.86:0/4293179523 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8be0095bf0 con 0x7f8bec073810 2024-03-22T19:09:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.144+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8bd40a0b10 msgr2=0x7f8bd40a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.144+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8bd40a0b10 0x7f8bd40a2fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bec074b10 tx=0x7f8bd8009260 comp rx=0 tx=0).stop 2024-03-22T19:09:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 msgr2=0x7f8bec199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 0x7f8bec199000 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8be000e9a0 tx=0x7f8be000ecb0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 shutdown_connections 2024-03-22T19:09:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8bd40a0b10 0x7f8bd40a2fd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bec108c00 0x7f8bec1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec073810 0x7f8bec199000 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 --2- 172.21.15.86:0/4293179523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec074df0 0x7f8bec199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 >> 172.21.15.86:0/4293179523 conn(0x7f8bec0fc2b0 msgr2=0x7f8bec104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 shutdown_connections 2024-03-22T19:09:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.145+0000 7f8bf3c00700 1 -- 172.21.15.86:0/4293179523 wait complete. 2024-03-22T19:09:19.169 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-22T19:09:19.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 -- 172.21.15.86:0/4187208388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 msgr2=0x7f0088073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 --2- 172.21.15.86:0/4187208388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 0x7f0088073c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0074009990 tx=0x7f0074009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 -- 172.21.15.86:0/4187208388 shutdown_connections 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 --2- 172.21.15.86:0/4187208388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0088108c00 0x7f008810b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 --2- 172.21.15.86:0/4187208388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 --2- 172.21.15.86:0/4187208388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 0x7f0088073c90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 -- 172.21.15.86:0/4187208388 >> 172.21.15.86:0/4187208388 conn(0x7f00880fc2b0 msgr2=0x7f00880fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 -- 172.21.15.86:0/4187208388 shutdown_connections 2024-03-22T19:09:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.365+0000 7f008ce00700 1 -- 172.21.15.86:0/4187208388 wait complete. 2024-03-22T19:09:19.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 Processor -- start 2024-03-22T19:09:19.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 -- start start 2024-03-22T19:09:19.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 0x7f0088198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0088108c00 0x7f00881a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00881a0a20 con 0x7f0088074df0 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00881a0b60 con 0x7f0088108c00 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.366+0000 7f008ce00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00881a0cf0 con 0x7f0088073810 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42908/0 (socket says 172.21.15.86:42908) 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 -- 172.21.15.86:0/1767912583 learned_addr learned my addr 172.21.15.86:0/1767912583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0086800700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0088108c00 0x7f00881a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085e00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 0x7f0088198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 -- 172.21.15.86:0/1767912583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 msgr2=0x7f0088198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 0x7f0088198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 -- 172.21.15.86:0/1767912583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0088108c00 msgr2=0x7f00881a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0088108c00 0x7f00881a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 -- 172.21.15.86:0/1767912583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740095f0 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f0085400700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088199480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f007400b680 tx=0x7f0074004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.367+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007401d070 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.368+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0074021410 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.368+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00881a0f70 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.368+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074017650 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.368+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00881a12e0 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.369+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00881079c0 con 0x7f0088074df0 2024-03-22T19:09:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.370+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0074021580 con 0x7f0088074df0 2024-03-22T19:09:19.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.372+0000 7f007ea00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f006c0a0a90 0x7f006c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.372+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00740cd310 con 0x7f0088074df0 2024-03-22T19:09:19.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.372+0000 7f0085e00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f006c0a0a90 0x7f006c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.372+0000 7f0085e00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f006c0a0a90 0x7f006c0a2f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0078009c60 tx=0x7f0078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.375+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0074096e20 con 0x7f0088074df0 2024-03-22T19:09:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] up:active 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1563353815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: pgmap v234: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-03-22T19:09:19.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:19 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/4293179523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-22T19:09:19.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] up:active 2024-03-22T19:09:19.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1563353815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-22T19:09:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-22T19:09:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-22T19:09:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-22T19:09:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-22T19:09:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: pgmap v234: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-03-22T19:09:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/4293179523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] up:active 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1563353815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-22T19:09:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: pgmap v234: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-03-22T19:09:19.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:19 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/4293179523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-22T19:09:19.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.619+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f00880611f0 con 0x7f006c0a0a90 2024-03-22T19:09:19.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.645+0000 7f007ea00700 1 -- 172.21.15.86:0/1767912583 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00880611f0 con 0x7f006c0a0a90 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.647+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f006c0a0a90 msgr2=0x7f006c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.647+0000 7f008ce00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f006c0a0a90 0x7f006c0a2f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0078009c60 tx=0x7f0078009260 comp rx=0 tx=0).stop 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.647+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 msgr2=0x7f0088199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.647+0000 7f008ce00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088199480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f007400b680 tx=0x7f0074004730 comp rx=0 tx=0).stop 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 shutdown_connections 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f006c0a0a90 0x7f006c0a2f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0088108c00 0x7f00881a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0088073810 0x7f0088198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 --2- 172.21.15.86:0/1767912583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0088074df0 0x7f0088199480 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 >> 172.21.15.86:0/1767912583 conn(0x7f00880fc2b0 msgr2=0x7f0088104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 shutdown_connections 2024-03-22T19:09:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.648+0000 7f008ce00700 1 -- 172.21.15.86:0/1767912583 wait complete. 2024-03-22T19:09:19.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 -- 172.21.15.86:0/4099569712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 msgr2=0x7f6eb8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 --2- 172.21.15.86:0/4099569712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8102390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40099c0 tx=0x7f6ea4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 -- 172.21.15.86:0/4099569712 shutdown_connections 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 --2- 172.21.15.86:0/4099569712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb81028d0 0x7f6eb8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 --2- 172.21.15.86:0/4099569712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 0x7f6eb8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 --2- 172.21.15.86:0/4099569712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8102390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 -- 172.21.15.86:0/4099569712 >> 172.21.15.86:0/4099569712 conn(0x7f6eb80fc2b0 msgr2=0x7f6eb80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 -- 172.21.15.86:0/4099569712 shutdown_connections 2024-03-22T19:09:19.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.864+0000 7f6ebea00700 1 -- 172.21.15.86:0/4099569712 wait complete. 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.865+0000 7f6ebea00700 1 Processor -- start 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.865+0000 7f6ebea00700 1 -- start start 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 0x7f6eb8198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb81028d0 0x7f6eb81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb81a0a20 con 0x7f6eb8101f30 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb81a0b60 con 0x7f6eb8100d40 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebea00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb81a0cf0 con 0x7f6eb81028d0 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 0x7f6eb8198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42952/0 (socket says 172.21.15.86:42952) 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 -- 172.21.15.86:0/3929773978 learned_addr learned my addr 172.21.15.86:0/3929773978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6ebca00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb81028d0 0x7f6eb81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 -- 172.21.15.86:0/3929773978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb81028d0 msgr2=0x7f6eb81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb81028d0 0x7f6eb81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 -- 172.21.15.86:0/3929773978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 msgr2=0x7f6eb8198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 0x7f6eb8198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 -- 172.21.15.86:0/3929773978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40095f0 con 0x7f6eb8101f30 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7e00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 0x7f6eb8198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.866+0000 7f6eb7400700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8199480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4003730 tx=0x7f6ea400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.867+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea401d070 con 0x7f6eb8101f30 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.867+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea400bda0 con 0x7f6eb8101f30 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.867+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb81a0f70 con 0x7f6eb8101f30 2024-03-22T19:09:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.867+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4004bd0 con 0x7f6eb8101f30 2024-03-22T19:09:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.867+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb81a1390 con 0x7f6eb8101f30 2024-03-22T19:09:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.868+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb8067ac0 con 0x7f6eb8101f30 2024-03-22T19:09:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.870+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6ea4004d30 con 0x7f6eb8101f30 2024-03-22T19:09:19.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.871+0000 7f6eb4c00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e9c0a0b10 0x7f6e9c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.871+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ea40cccf0 con 0x7f6eb8101f30 2024-03-22T19:09:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.871+0000 7f6eb7e00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e9c0a0b10 0x7f6e9c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.871+0000 7f6eb7e00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e9c0a0b10 0x7f6e9c0a2fd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8101d90 tx=0x7f6ea8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:19.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:19.874+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6ea4096800 con 0x7f6eb8101f30 2024-03-22T19:09:20.008 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:09:19.625+0000 7fdd90400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-22T19:09:20.009 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:09:19.625+0000 7fdd90400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-22T19:09:20.009 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:09:19.625+0000 7fdd90400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-22T19:09:20.009 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:09:19.625+0000 7fdd90400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-22T19:09:20.009 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:09:19 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:09:19.626+0000 7fdd90400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-22T19:09:20.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.117+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb8074700 con 0x7f6e9c0a0b10 2024-03-22T19:09:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.140+0000 7f6eb4c00700 1 -- 172.21.15.86:0/3929773978 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6eb8074700 con 0x7f6e9c0a0b10 2024-03-22T19:09:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e9c0a0b10 msgr2=0x7f6e9c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e9c0a0b10 0x7f6e9c0a2fd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8101d90 tx=0x7f6ea8009260 comp rx=0 tx=0).stop 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 msgr2=0x7f6eb8199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8199480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4003730 tx=0x7f6ea400fa00 comp rx=0 tx=0).stop 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 shutdown_connections 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e9c0a0b10 0x7f6e9c0a2fd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb81028d0 0x7f6eb81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8100d40 0x7f6eb8198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 --2- 172.21.15.86:0/3929773978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb8101f30 0x7f6eb8199480 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 >> 172.21.15.86:0/3929773978 conn(0x7f6eb80fc2b0 msgr2=0x7f6eb8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 shutdown_connections 2024-03-22T19:09:20.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.143+0000 7f6ebea00700 1 -- 172.21.15.86:0/3929773978 wait complete. 2024-03-22T19:09:20.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-22T19:09:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 -- 172.21.15.86:0/329366616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c101f40 msgr2=0x7f610c1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 --2- 172.21.15.86:0/329366616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c101f40 0x7f610c1023a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60f80099c0 tx=0x7f60f8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 -- 172.21.15.86:0/329366616 shutdown_connections 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 --2- 172.21.15.86:0/329366616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610c1028e0 0x7f610c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 --2- 172.21.15.86:0/329366616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c100d50 0x7f610c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 --2- 172.21.15.86:0/329366616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c101f40 0x7f610c1023a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 -- 172.21.15.86:0/329366616 >> 172.21.15.86:0/329366616 conn(0x7f610c0fc2c0 msgr2=0x7f610c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 -- 172.21.15.86:0/329366616 shutdown_connections 2024-03-22T19:09:20.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.364+0000 7f6114600700 1 -- 172.21.15.86:0/329366616 wait complete. 2024-03-22T19:09:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 Processor -- start 2024-03-22T19:09:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 -- start start 2024-03-22T19:09:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 0x7f610c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610c101f40 0x7f610c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c1028e0 0x7f610c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c1a0a30 con 0x7f610c100d50 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c1a0b70 con 0x7f610c101f40 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6114600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c1a0d00 con 0x7f610c1028e0 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6111c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 0x7f610c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.365+0000 7f6111c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 0x7f610c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42958/0 (socket says 172.21.15.86:42958) 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 -- 172.21.15.86:0/3203524079 learned_addr learned my addr 172.21.15.86:0/3203524079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111200700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610c101f40 0x7f610c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6112600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c1028e0 0x7f610c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 -- 172.21.15.86:0/3203524079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c1028e0 msgr2=0x7f610c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c1028e0 0x7f610c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 -- 172.21.15.86:0/3203524079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610c101f40 msgr2=0x7f610c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610c101f40 0x7f610c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 -- 172.21.15.86:0/3203524079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f80095f0 con 0x7f610c100d50 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6112600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c1028e0 0x7f610c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:09:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6111c00700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 0x7f610c198f00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f610000e970 tx=0x7f610000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100009790 con 0x7f610c100d50 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61000044e0 con 0x7f610c100d50 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610c1a0fe0 con 0x7f610c100d50 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100003e30 con 0x7f610c100d50 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.366+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610c1a1410 con 0x7f610c100d50 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.367+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610c067ac0 con 0x7f610c100d50 2024-03-22T19:09:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.369+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6100003f90 con 0x7f610c100d50 2024-03-22T19:09:20.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.371+0000 7f610a000700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f60f40a0b10 0x7f60f40a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.371+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6100014070 con 0x7f610c100d50 2024-03-22T19:09:20.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.371+0000 7f6111200700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f60f40a0b10 0x7f60f40a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.371+0000 7f6111200700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f60f40a0b10 0x7f60f40a2fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b580 tx=0x7f60f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.375+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6100095cc0 con 0x7f610c100d50 2024-03-22T19:09:20.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] up:active 2024-03-22T19:09:20.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/4293179523' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-22T19:09:20.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-22T19:09:20.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-22T19:09:20.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-22T19:09:20.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-22T19:09:20.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:20 smithi131 ceph-mon[78559]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:20.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] up:active 2024-03-22T19:09:20.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/4293179523' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-22T19:09:20.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-22T19:09:20.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-22T19:09:20.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-22T19:09:20.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-22T19:09:20.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:20 smithi086 ceph-mon[38032]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] up:active 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/4293179523' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-22T19:09:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:20 smithi167 ceph-mon[47728]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:20.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.621+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f610c1a1690 con 0x7f60f40a0b10 2024-03-22T19:09:20.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.625+0000 7f610a000700 1 -- 172.21.15.86:0/3203524079 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f610c1a1690 con 0x7f60f40a0b10 2024-03-22T19:09:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/7ce3a1cb-dec7-4df1-acff-164eb4bf43ab 2024-03-22T19:09:20.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.627+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f60f40a0b10 msgr2=0x7f60f40a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.627+0000 7f6114600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f60f40a0b10 0x7f60f40a2fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b580 tx=0x7f60f801a040 comp rx=0 tx=0).stop 2024-03-22T19:09:20.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 msgr2=0x7f610c198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 0x7f610c198f00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f610000e970 tx=0x7f610000ed30 comp rx=0 tx=0).stop 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 shutdown_connections 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f60f40a0b10 0x7f60f40a2fd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f610c1028e0 0x7f610c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610c100d50 0x7f610c198f00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 --2- 172.21.15.86:0/3203524079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610c101f40 0x7f610c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 >> 172.21.15.86:0/3203524079 conn(0x7f610c0fc2c0 msgr2=0x7f610c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 shutdown_connections 2024-03-22T19:09:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.628+0000 7f6114600700 1 -- 172.21.15.86:0/3203524079 wait complete. 2024-03-22T19:09:20.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-22T19:09:20.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.842+0000 7f0c2c600700 1 -- 172.21.15.86:0/2502397656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c241028d0 msgr2=0x7f0c24109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.842+0000 7f0c2c600700 1 --2- 172.21.15.86:0/2502397656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c241028d0 0x7f0c24109180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00b0d0 tx=0x7f0c1c00b3e0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 -- 172.21.15.86:0/2502397656 shutdown_connections 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 --2- 172.21.15.86:0/2502397656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c241028d0 0x7f0c24109180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 --2- 172.21.15.86:0/2502397656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c24100d40 0x7f0c24101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 --2- 172.21.15.86:0/2502397656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c24101f30 0x7f0c24102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 -- 172.21.15.86:0/2502397656 >> 172.21.15.86:0/2502397656 conn(0x7f0c240fc2b0 msgr2=0x7f0c240fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 -- 172.21.15.86:0/2502397656 shutdown_connections 2024-03-22T19:09:20.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.843+0000 7f0c2c600700 1 -- 172.21.15.86:0/2502397656 wait complete. 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 Processor -- start 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 -- start start 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 0x7f0c24074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c24101f30 0x7f0c240731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c241028d0 0x7f0c240736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24073e80 con 0x7f0c24100d40 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24073ff0 con 0x7f0c241028d0 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c2c600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c241a0e20 con 0x7f0c24101f30 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c29c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 0x7f0c24074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c29c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 0x7f0c24074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42976/0 (socket says 172.21.15.86:42976) 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c29c00700 1 -- 172.21.15.86:0/3652012660 learned_addr learned my addr 172.21.15.86:0/3652012660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:20.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.844+0000 7f0c29200700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c24101f30 0x7f0c240731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c2a600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c241028d0 0x7f0c240736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c29c00700 1 -- 172.21.15.86:0/3652012660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c24101f30 msgr2=0x7f0c240731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c29c00700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c24101f30 0x7f0c240731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c29c00700 1 -- 172.21.15.86:0/3652012660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c241028d0 msgr2=0x7f0c240736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c29c00700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c241028d0 0x7f0c240736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c29c00700 1 -- 172.21.15.86:0/3652012660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c009d00 con 0x7f0c24100d40 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c29c00700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 0x7f0c24074b10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800d980 tx=0x7f0c1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800b8f0 con 0x7f0c24100d40 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c18007620 con 0x7f0c24100d40 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c241a10b0 con 0x7f0c24100d40 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800f3c0 con 0x7f0c24100d40 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.845+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c241a1610 con 0x7f0c24100d40 2024-03-22T19:09:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.846+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c241080f0 con 0x7f0c24100d40 2024-03-22T19:09:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.848+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0c180105f0 con 0x7f0c24100d40 2024-03-22T19:09:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.849+0000 7f0c22000700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0c0c0a0b10 0x7f0c0c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.849+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c180cbe00 con 0x7f0c24100d40 2024-03-22T19:09:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.850+0000 7f0c29200700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0c0c0a0b10 0x7f0c0c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.850+0000 7f0c29200700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0c0c0a0b10 0x7f0c0c0a2fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009be0 tx=0x7f0c10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:20.853+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c18095880 con 0x7f0c24100d40 2024-03-22T19:09:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.100+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c240611f0 con 0x7f0c0c0a0b10 2024-03-22T19:09:21.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.127+0000 7f0c22000700 1 -- 172.21.15.86:0/3652012660 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0c240611f0 con 0x7f0c0c0a0b10 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.129+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0c0c0a0b10 msgr2=0x7f0c0c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.129+0000 7f0c2c600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0c0c0a0b10 0x7f0c0c0a2fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009be0 tx=0x7f0c10009260 comp rx=0 tx=0).stop 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.129+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 msgr2=0x7f0c24074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.129+0000 7f0c2c600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 0x7f0c24074b10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800d980 tx=0x7f0c1800dc90 comp rx=0 tx=0).stop 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 shutdown_connections 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0c0c0a0b10 0x7f0c0c0a2fd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c241028d0 0x7f0c240736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c24100d40 0x7f0c24074b10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 --2- 172.21.15.86:0/3652012660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c24101f30 0x7f0c240731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 >> 172.21.15.86:0/3652012660 conn(0x7f0c240fc2b0 msgr2=0x7f0c24105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 shutdown_connections 2024-03-22T19:09:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.130+0000 7f0c2c600700 1 -- 172.21.15.86:0/3652012660 wait complete. 2024-03-22T19:09:21.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 -- 172.21.15.86:0/2284111074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 msgr2=0x7f9f54103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2284111074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54103ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009990 tx=0x7f9f48009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 -- 172.21.15.86:0/2284111074 shutdown_connections 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2284111074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 0x7f9f54108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2284111074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54103ba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2284111074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f541040e0 0x7f9f541064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 -- 172.21.15.86:0/2284111074 >> 172.21.15.86:0/2284111074 conn(0x7f9f540fb1a0 msgr2=0x7f9f540fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 -- 172.21.15.86:0/2284111074 shutdown_connections 2024-03-22T19:09:21.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.349+0000 7f9f5c600700 1 -- 172.21.15.86:0/2284111074 wait complete. 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.350+0000 7f9f5c600700 1 Processor -- start 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.350+0000 7f9f5c600700 1 -- start start 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5c600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f541040e0 0x7f9f54199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5c600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 0x7f9f541a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541a0a30 con 0x7f9f541017b0 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5c600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541a0b70 con 0x7f9f54106a10 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5c600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541a0d00 con 0x7f9f541040e0 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42996/0 (socket says 172.21.15.86:42996) 2024-03-22T19:09:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 -- 172.21.15.86:0/2098270225 learned_addr learned my addr 172.21.15.86:0/2098270225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5a600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 0x7f9f541a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59200700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f541040e0 0x7f9f54199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 -- 172.21.15.86:0/2098270225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f541040e0 msgr2=0x7f9f54199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f541040e0 0x7f9f54199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 -- 172.21.15.86:0/2098270225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 msgr2=0x7f9f541a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 0x7f9f541a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f59c00700 1 -- 172.21.15.86:0/2098270225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480095f0 con 0x7f9f541017b0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.351+0000 7f9f5a600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 0x7f9f541a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.352+0000 7f9f59c00700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54198f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9f48000c00 tx=0x7f9f48005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.352+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4801d070 con 0x7f9f541017b0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.352+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48004ca0 con 0x7f9f541017b0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.352+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541a0f80 con 0x7f9f541017b0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.352+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480055f0 con 0x7f9f541017b0 2024-03-22T19:09:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.352+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a12f0 con 0x7f9f541017b0 2024-03-22T19:09:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.353+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f541085e0 con 0x7f9f541017b0 2024-03-22T19:09:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.355+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f9f4800fc40 con 0x7f9f541017b0 2024-03-22T19:09:21.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.356+0000 7f9f52000700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f3c0a0b20 0x7f9f3c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.356+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9f480cd3e0 con 0x7f9f541017b0 2024-03-22T19:09:21.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.356+0000 7f9f59200700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f3c0a0b20 0x7f9f3c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.357+0000 7f9f59200700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f3c0a0b20 0x7f9f3c0a2fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009bf0 tx=0x7f9f40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:21.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.360+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9f48096d90 con 0x7f9f541017b0 2024-03-22T19:09:21.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:21.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: pgmap v235: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s wr, 13 op/s 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] up:active 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: Cluster is now healthy 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] up:active 2024-03-22T19:09:21.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:21 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:09:21.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:21.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: pgmap v235: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s wr, 13 op/s 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] up:active 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: Cluster is now healthy 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] up:active 2024-03-22T19:09:21.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:21 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:09:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: pgmap v235: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s wr, 13 op/s 2024-03-22T19:09:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] up:active 2024-03-22T19:09:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-22T19:09:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T19:09:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: Cluster is now healthy 2024-03-22T19:09:21.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-03-22T19:09:21.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-22T19:09:21.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-22T19:09:21.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:21.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] up:active 2024-03-22T19:09:21.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:21 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:09:21.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.603+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f54067a90 con 0x7f9f3c0a0b20 2024-03-22T19:09:21.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.607+0000 7f9f52000700 1 -- 172.21.15.86:0/2098270225 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f9f54067a90 con 0x7f9f3c0a0b20 2024-03-22T19:09:21.608 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/4e72917a-ce51-40c9-b401-83d72303e636 2024-03-22T19:09:21.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f3c0a0b20 msgr2=0x7f9f3c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f3c0a0b20 0x7f9f3c0a2fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009bf0 tx=0x7f9f40009260 comp rx=0 tx=0).stop 2024-03-22T19:09:21.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 msgr2=0x7f9f54198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54198f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9f48000c00 tx=0x7f9f48005ef0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 shutdown_connections 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f3c0a0b20 0x7f9f3c0a2fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106a10 0x7f9f541a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f541017b0 0x7f9f54198f00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 --2- 172.21.15.86:0/2098270225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f541040e0 0x7f9f54199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 >> 172.21.15.86:0/2098270225 conn(0x7f9f540fb1a0 msgr2=0x7f9f54100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 shutdown_connections 2024-03-22T19:09:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.610+0000 7f9f5c600700 1 -- 172.21.15.86:0/2098270225 wait complete. 2024-03-22T19:09:21.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:09:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 -- 172.21.15.86:0/4017328303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4100c50 msgr2=0x7fb6c4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 --2- 172.21.15.86:0/4017328303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4100c50 0x7fb6c4101070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40099c0 tx=0x7fb6b4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 -- 172.21.15.86:0/4017328303 shutdown_connections 2024-03-22T19:09:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 --2- 172.21.15.86:0/4017328303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41027e0 0x7fb6c4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 --2- 172.21.15.86:0/4017328303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4100c50 0x7fb6c4101070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 --2- 172.21.15.86:0/4017328303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c4101e40 0x7fb6c41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 -- 172.21.15.86:0/4017328303 >> 172.21.15.86:0/4017328303 conn(0x7fb6c40fc1c0 msgr2=0x7fb6c40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:21.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 -- 172.21.15.86:0/4017328303 shutdown_connections 2024-03-22T19:09:21.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.820+0000 7fb6c9800700 1 -- 172.21.15.86:0/4017328303 wait complete. 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.821+0000 7fb6c9800700 1 Processor -- start 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.821+0000 7fb6c9800700 1 -- start start 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c9800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c4100c50 0x7fb6c4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 0x7fb6c4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c9800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41027e0 0x7fb6c41a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c9800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a0930 con 0x7fb6c4101e40 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c9800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a0a70 con 0x7fb6c41027e0 2024-03-22T19:09:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c9800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a0c00 con 0x7fb6c4100c50 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 0x7fb6c4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 0x7fb6c4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43030/0 (socket says 172.21.15.86:43030) 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 -- 172.21.15.86:0/3665107112 learned_addr learned my addr 172.21.15.86:0/3665107112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2a00700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c4100c50 0x7fb6c4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c3400700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41027e0 0x7fb6c41a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 -- 172.21.15.86:0/3665107112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c4100c50 msgr2=0x7fb6c4198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c4100c50 0x7fb6c4198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 -- 172.21.15.86:0/3665107112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41027e0 msgr2=0x7fb6c41a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41027e0 0x7fb6c41a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.822+0000 7fb6c2000700 1 -- 172.21.15.86:0/3665107112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b40095f0 con 0x7fb6c4101e40 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.823+0000 7fb6c2000700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 0x7fb6c4199340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e9a0 tx=0x7fb6b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.823+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0009790 con 0x7fb6c4101e40 2024-03-22T19:09:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.823+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b0010430 con 0x7fb6c4101e40 2024-03-22T19:09:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.823+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c41a0ee0 con 0x7fb6c4101e40 2024-03-22T19:09:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.823+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0018680 con 0x7fb6c4101e40 2024-03-22T19:09:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.823+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41a13b0 con 0x7fb6c4101e40 2024-03-22T19:09:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.824+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c40679e0 con 0x7fb6c4101e40 2024-03-22T19:09:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.826+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fb6b00105a0 con 0x7fb6c4101e40 2024-03-22T19:09:21.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.827+0000 7fb6bb400700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6a80a0ba0 0x7fb6a80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:21.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.827+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6b0014070 con 0x7fb6c4101e40 2024-03-22T19:09:21.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.827+0000 7fb6c2a00700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6a80a0ba0 0x7fb6a80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:21.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.828+0000 7fb6c2a00700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6a80a0ba0 0x7fb6a80a3060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4003730 tx=0x7fb6b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:21.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:21.829+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb6b0095d90 con 0x7fb6c4101e40 2024-03-22T19:09:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.080+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6c41a1710 con 0x7fb6c4101e40 2024-03-22T19:09:22.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.081+0000 7fb6bb400700 1 -- 172.21.15.86:0/3665107112 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10924 (secure 0 0 0) 0x7fb6b0026020 con 0x7fb6c4101e40 2024-03-22T19:09:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:22.082 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:09:21.111792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6a80a0ba0 msgr2=0x7fb6a80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6a80a0ba0 0x7fb6a80a3060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4003730 tx=0x7fb6b4000bc0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 msgr2=0x7fb6c4199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 0x7fb6c4199340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e9a0 tx=0x7fb6b000ecb0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 shutdown_connections 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6a80a0ba0 0x7fb6a80a3060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41027e0 0x7fb6c41a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c4100c50 0x7fb6c4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 --2- 172.21.15.86:0/3665107112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c4101e40 0x7fb6c4199340 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 >> 172.21.15.86:0/3665107112 conn(0x7fb6c40fc1c0 msgr2=0x7fb6c4107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 shutdown_connections 2024-03-22T19:09:22.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.083+0000 7fb6c9800700 1 -- 172.21.15.86:0/3665107112 wait complete. 2024-03-22T19:09:22.086 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:09:22.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 -- 172.21.15.86:0/3064608544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 msgr2=0x7f21401022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 --2- 172.21.15.86:0/3064608544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f21401022a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f212c0099c0 tx=0x7f212c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 -- 172.21.15.86:0/3064608544 shutdown_connections 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 --2- 172.21.15.86:0/3064608544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21401027e0 0x7f2140109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 --2- 172.21.15.86:0/3064608544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2140100c50 0x7f2140101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 --2- 172.21.15.86:0/3064608544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f21401022a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 -- 172.21.15.86:0/3064608544 >> 172.21.15.86:0/3064608544 conn(0x7f21400fc1c0 msgr2=0x7f21400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 -- 172.21.15.86:0/3064608544 shutdown_connections 2024-03-22T19:09:22.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.295+0000 7f2148400700 1 -- 172.21.15.86:0/3064608544 wait complete. 2024-03-22T19:09:22.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.296+0000 7f2148400700 1 Processor -- start 2024-03-22T19:09:22.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.296+0000 7f2148400700 1 -- start start 2024-03-22T19:09:22.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.296+0000 7f2148400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2140100c50 0x7f2140196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2148400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f2140197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2148400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21401027e0 0x7f214019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2148400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019e6f0 con 0x7f2140101e40 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2148400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019e830 con 0x7f2140100c50 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2148400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019e9c0 con 0x7f21401027e0 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f2140197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2140100c50 0x7f2140196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f2140197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43046/0 (socket says 172.21.15.86:43046) 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 -- 172.21.15.86:0/3725279498 learned_addr learned my addr 172.21.15.86:0/3725279498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2146400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21401027e0 0x7f214019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 -- 172.21.15.86:0/3725279498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21401027e0 msgr2=0x7f214019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21401027e0 0x7f214019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 -- 172.21.15.86:0/3725279498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2140100c50 msgr2=0x7f2140196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2140100c50 0x7f2140196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 -- 172.21.15.86:0/3725279498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c0095f0 con 0x7f2140101e40 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.297+0000 7f2145000700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f2140197100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.298+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c01d070 con 0x7f2140101e40 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.298+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212c005030 con 0x7f2140101e40 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.298+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c004bb0 con 0x7f2140101e40 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.298+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214019ec40 con 0x7f2140101e40 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.298+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214019f060 con 0x7f2140101e40 2024-03-22T19:09:22.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.299+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140067a50 con 0x7f2140101e40 2024-03-22T19:09:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.301+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f212c0051a0 con 0x7f2140101e40 2024-03-22T19:09:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.302+0000 7f213e000700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f21280a0ba0 0x7f21280a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.302+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f212c0ccdb0 con 0x7f2140101e40 2024-03-22T19:09:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.302+0000 7f2145a00700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f21280a0ba0 0x7f21280a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.303+0000 7f2145a00700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f21280a0ba0 0x7f21280a3060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2140101ca0 tx=0x7f2134009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.306+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2140067a50 con 0x7f2140101e40 2024-03-22T19:09:22.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:22 smithi131 ceph-mon[78559]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:22.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:22 smithi131 ceph-mon[78559]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-22T19:09:22.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:22 smithi131 ceph-mon[78559]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:22.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:22 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3665107112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:22.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:22 smithi086 ceph-mon[38032]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:22.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:22 smithi086 ceph-mon[38032]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-22T19:09:22.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:22 smithi086 ceph-mon[38032]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:22.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:22 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3665107112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.549+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f214002ccb0 con 0x7f2140101e40 2024-03-22T19:09:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.550+0000 7f213e000700 1 -- 172.21.15.86:0/3725279498 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f212c0a0030 con 0x7f2140101e40 2024-03-22T19:09:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:22.552 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:14.621457+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.552+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f21280a0ba0 msgr2=0x7f21280a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.552+0000 7f2148400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f21280a0ba0 0x7f21280a3060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2140101ca0 tx=0x7f2134009260 comp rx=0 tx=0).stop 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.552+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 msgr2=0x7f2140197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.552+0000 7f2148400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f2140197100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c00f690 comp rx=0 tx=0).stop 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 shutdown_connections 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f21280a0ba0 0x7f21280a3060 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21401027e0 0x7f214019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2140100c50 0x7f2140196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 --2- 172.21.15.86:0/3725279498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2140101e40 0x7f2140197100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 >> 172.21.15.86:0/3725279498 conn(0x7f21400fc1c0 msgr2=0x7f2140105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 shutdown_connections 2024-03-22T19:09:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.553+0000 7f2148400700 1 -- 172.21.15.86:0/3725279498 wait complete. 2024-03-22T19:09:22.573 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T19:09:22.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T19:09:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:22 smithi167 ceph-mon[47728]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:22 smithi167 ceph-mon[47728]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-22T19:09:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:22 smithi167 ceph-mon[47728]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:09:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:22 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3665107112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 -- 172.21.15.86:0/962706507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 msgr2=0x7fbfbc109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 --2- 172.21.15.86:0/962706507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 0x7fbfbc109090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009f90 tx=0x7fbfb400b330 comp rx=0 tx=0).stop 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 -- 172.21.15.86:0/962706507 shutdown_connections 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 --2- 172.21.15.86:0/962706507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 0x7fbfbc109090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 --2- 172.21.15.86:0/962706507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 0x7fbfbc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 --2- 172.21.15.86:0/962706507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 -- 172.21.15.86:0/962706507 >> 172.21.15.86:0/962706507 conn(0x7fbfbc0fc1c0 msgr2=0x7fbfbc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 -- 172.21.15.86:0/962706507 shutdown_connections 2024-03-22T19:09:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.765+0000 7fbfc1200700 1 -- 172.21.15.86:0/962706507 wait complete. 2024-03-22T19:09:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.766+0000 7fbfc1200700 1 Processor -- start 2024-03-22T19:09:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.766+0000 7fbfc1200700 1 -- start start 2024-03-22T19:09:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.766+0000 7fbfc1200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 0x7fbfbc198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.766+0000 7fbfc1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.766+0000 7fbfc1200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 0x7fbfbc1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfc1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1a0a50 con 0x7fbfbc101e40 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfc1200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1a0b90 con 0x7fbfbc100c50 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfc1200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1a0d20 con 0x7fbfbc1027e0 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43062/0 (socket says 172.21.15.86:43062) 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 -- 172.21.15.86:0/2387470865 learned_addr learned my addr 172.21.15.86:0/2387470865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9e00700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 0x7fbfbc198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfba800700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 0x7fbfbc1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 -- 172.21.15.86:0/2387470865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 msgr2=0x7fbfbc1a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 0x7fbfbc1a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 -- 172.21.15.86:0/2387470865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 msgr2=0x7fbfbc198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 0x7fbfbc198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 -- 172.21.15.86:0/2387470865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80095f0 con 0x7fbfbc101e40 2024-03-22T19:09:22.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9e00700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 0x7fbfbc198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.767+0000 7fbfb9400700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc199460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800ec10 tx=0x7fbfa800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.768+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cc90 con 0x7fbfbc101e40 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.768+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfa80044e0 con 0x7fbfbc101e40 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.768+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4009c40 con 0x7fbfbc101e40 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.768+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8010430 con 0x7fbfbc101e40 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.768+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc1a12f0 con 0x7fbfbc101e40 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.769+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc0679e0 con 0x7fbfbc101e40 2024-03-22T19:09:22.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.770+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fbfa8003fb0 con 0x7fbfbc101e40 2024-03-22T19:09:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.772+0000 7fbfb2a00700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbfa00a0b20 0x7fbfa00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.772+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbfa8014070 con 0x7fbfbc101e40 2024-03-22T19:09:22.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.772+0000 7fbfb9e00700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbfa00a0b20 0x7fbfa00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.772+0000 7fbfb9e00700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbfa00a0b20 0x7fbfa00a2fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc101ca0 tx=0x7fbfac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:22.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.775+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbfa8095f60 con 0x7fbfbc101e40 2024-03-22T19:09:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.980+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfbc074700 con 0x7fbfa00a0b20 2024-03-22T19:09:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.992+0000 7fbfb2a00700 1 -- 172.21.15.86:0/2387470865 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+331787 (secure 0 0 0) 0x7fbfbc074700 con 0x7fbfa00a0b20 2024-03-22T19:09:22.999 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbfa00a0b20 msgr2=0x7fbfa00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbfa00a0b20 0x7fbfa00a2fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc101ca0 tx=0x7fbfac009260 comp rx=0 tx=0).stop 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 msgr2=0x7fbfbc199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc199460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800ec10 tx=0x7fbfa800ef20 comp rx=0 tx=0).stop 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 shutdown_connections 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbfa00a0b20 0x7fbfa00a2fe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfbc1027e0 0x7fbfbc1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfbc100c50 0x7fbfbc198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 --2- 172.21.15.86:0/2387470865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfbc101e40 0x7fbfbc199460 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 >> 172.21.15.86:0/2387470865 conn(0x7fbfbc0fc1c0 msgr2=0x7fbfbc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 shutdown_connections 2024-03-22T19:09:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:22.997+0000 7fbfc1200700 1 -- 172.21.15.86:0/2387470865 wait complete. 2024-03-22T19:09:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:09:23.017 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-03-22T19:09:22.150841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597853,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":136,"num_read_kb":117,"num_write":351,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334688,"kb_used_data":10484,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124607136,"statfs":{"total":1151940427776,"available":1151597707264,"internally_reserved":0,"allocated":10735616,"data_stored":6742869,"data_compressed":106959,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7485,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":170,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.529368"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804515+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:15.804515+0000","last_peered":"2024-03-22T19:09:15.804515+0000","last_clean":"2024-03-22T19:09:15.804515+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:15.804515+0000","last_undegraded":"2024-03-22T19:09:15.804515+0000","last_fullsized":"2024-03-22T19:09:15.804515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957320+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:14.957320+0000","last_peered":"2024-03-22T19:09:14.957320+0000","last_clean":"2024-03-22T19:09:14.957320+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:14.957320+0000","last_undegraded":"2024-03-22T19:09:14.957320+0000","last_fullsized":"2024-03-22T19:09:14.957320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957385+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:14.957385+0000","last_peered":"2024-03-22T19:09:14.957385+0000","last_clean":"2024-03-22T19:09:14.957385+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:14.957385+0000","last_undegraded":"2024-03-22T19:09:14.957385+0000","last_fullsized":"2024-03-22T19:09:14.957385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.093072+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:15.093072+0000","last_peered":"2024-03-22T19:09:15.093072+0000","last_clean":"2024-03-22T19:09:15.093072+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:15.093072+0000","last_undegraded":"2024-03-22T19:09:15.093072+0000","last_fullsized":"2024-03-22T19:09:15.093072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643954+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:14.643954+0000","last_peered":"2024-03-22T19:09:14.643954+0000","last_clean":"2024-03-22T19:09:14.643954+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:14.643954+0000","last_undegraded":"2024-03-22T19:09:14.643954+0000","last_fullsized":"2024-03-22T19:09:14.643954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.140787+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:17.140787+0000","last_peered":"2024-03-22T19:09:17.140787+0000","last_clean":"2024-03-22T19:09:17.140787+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:17.140787+0000","last_undegraded":"2024-03-22T19:09:17.140787+0000","last_fullsized":"2024-03-22T19:09:17.140787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630682+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:14.630682+0000","last_peered":"2024-03-22T19:09:14.630682+0000","last_clean":"2024-03-22T19:09:14.630682+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:14.630682+0000","last_undegraded":"2024-03-22T19:09:14.630682+0000","last_fullsized":"2024-03-22T19:09:14.630682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530409+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:16.530409+0000","last_peered":"2024-03-22T19:09:16.530409+0000","last_clean":"2024-03-22T19:09:16.530409+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:16.530409+0000","last_undegraded":"2024-03-22T19:09:16.530409+0000","last_fullsized":"2024-03-22T19:09:16.530409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.644746+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:13.644746+0000","last_peered":"2024-03-22T19:09:13.644746+0000","last_clean":"2024-03-22T19:09:13.644746+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:13.644746+0000","last_undegraded":"2024-03-22T19:09:13.644746+0000","last_fullsized":"2024-03-22T19:09:13.644746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.141158+0000","last_change":"2024-03-22T19:09:13.651761+0000","last_active":"2024-03-22T19:09:17.141158+0000","last_peered":"2024-03-22T19:09:17.141158+0000","last_clean":"2024-03-22T19:09:17.141158+0000","last_became_active":"2024-03-22T19:09:13.651374+0000","last_became_peered":"2024-03-22T19:09:13.651374+0000","last_unstale":"2024-03-22T19:09:17.141158+0000","last_undegraded":"2024-03-22T19:09:17.141158+0000","last_fullsized":"2024-03-22T19:09:17.141158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:08:41.358335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631127+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:14.631127+0000","last_peered":"2024-03-22T19:09:14.631127+0000","last_clean":"2024-03-22T19:09:14.631127+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:14.631127+0000","last_undegraded":"2024-03-22T19:09:14.631127+0000","last_fullsized":"2024-03-22T19:09:14.631127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.093061+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:15.093061+0000","last_peered":"2024-03-22T19:09:15.093061+0000","last_clean":"2024-03-22T19:09:15.093061+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:15.093061+0000","last_undegraded":"2024-03-22T19:09:15.093061+0000","last_fullsized":"2024-03-22T19:09:15.093061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527806+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:14.527806+0000","last_peered":"2024-03-22T19:09:14.527806+0000","last_clean":"2024-03-22T19:09:14.527806+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:14.527806+0000","last_undegraded":"2024-03-22T19:09:14.527806+0000","last_fullsized":"2024-03-22T19:09:14.527806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957204+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:14.957204+0000","last_peered":"2024-03-22T19:09:14.957204+0000","last_clean":"2024-03-22T19:09:14.957204+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:14.957204+0000","last_undegraded":"2024-03-22T19:09:14.957204+0000","last_fullsized":"2024-03-22T19:09:14.957204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957125+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:14.957125+0000","last_peered":"2024-03-22T19:09:14.957125+0000","last_clean":"2024-03-22T19:09:14.957125+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:14.957125+0000","last_undegraded":"2024-03-22T19:09:14.957125+0000","last_fullsized":"2024-03-22T19:09:14.957125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.645969+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:13.645969+0000","last_peered":"2024-03-22T19:09:13.645969+0000","last_clean":"2024-03-22T19:09:13.645969+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:13.645969+0000","last_undegraded":"2024-03-22T19:09:13.645969+0000","last_fullsized":"2024-03-22T19:09:13.645969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804526+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:15.804526+0000","last_peered":"2024-03-22T19:09:15.804526+0000","last_clean":"2024-03-22T19:09:15.804526+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:15.804526+0000","last_undegraded":"2024-03-22T19:09:15.804526+0000","last_fullsized":"2024-03-22T19:09:15.804526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.111027+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:16.111027+0000","last_peered":"2024-03-22T19:09:16.111027+0000","last_clean":"2024-03-22T19:09:16.111027+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:16.111027+0000","last_undegraded":"2024-03-22T19:09:16.111027+0000","last_fullsized":"2024-03-22T19:09:16.111027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924829+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:14.924829+0000","last_peered":"2024-03-22T19:09:14.924829+0000","last_clean":"2024-03-22T19:09:14.924829+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:14.924829+0000","last_undegraded":"2024-03-22T19:09:14.924829+0000","last_fullsized":"2024-03-22T19:09:14.924829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.166749+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:09:20.166749+0000","last_peered":"2024-03-22T19:09:20.166749+0000","last_clean":"2024-03-22T19:09:20.166749+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:09:20.166749+0000","last_undegraded":"2024-03-22T19:09:20.166749+0000","last_fullsized":"2024-03-22T19:09:20.166749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530460+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:16.530460+0000","last_peered":"2024-03-22T19:09:16.530460+0000","last_clean":"2024-03-22T19:09:16.530460+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:16.530460+0000","last_undegraded":"2024-03-22T19:09:16.530460+0000","last_fullsized":"2024-03-22T19:09:16.530460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.144432+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:17.144432+0000","last_peered":"2024-03-22T19:09:17.144432+0000","last_clean":"2024-03-22T19:09:17.144432+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:17.144432+0000","last_undegraded":"2024-03-22T19:09:17.144432+0000","last_fullsized":"2024-03-22T19:09:17.144432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957612+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:14.957612+0000","last_peered":"2024-03-22T19:09:14.957612+0000","last_clean":"2024-03-22T19:09:14.957612+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:14.957612+0000","last_undegraded":"2024-03-22T19:09:14.957612+0000","last_fullsized":"2024-03-22T19:09:14.957612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.153511+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:19.153511+0000","last_peered":"2024-03-22T19:09:19.153511+0000","last_clean":"2024-03-22T19:09:19.153511+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:19.153511+0000","last_undegraded":"2024-03-22T19:09:19.153511+0000","last_fullsized":"2024-03-22T19:09:19.153511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.647575+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:13.647575+0000","last_peered":"2024-03-22T19:09:13.647575+0000","last_clean":"2024-03-22T19:09:13.647575+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:13.647575+0000","last_undegraded":"2024-03-22T19:09:13.647575+0000","last_fullsized":"2024-03-22T19:09:13.647575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643981+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:14.643981+0000","last_peered":"2024-03-22T19:09:14.643981+0000","last_clean":"2024-03-22T19:09:14.643981+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:14.643981+0000","last_undegraded":"2024-03-22T19:09:14.643981+0000","last_fullsized":"2024-03-22T19:09:14.643981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.943219+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:14.943219+0000","last_peered":"2024-03-22T19:09:14.943219+0000","last_clean":"2024-03-22T19:09:14.943219+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:14.943219+0000","last_undegraded":"2024-03-22T19:09:14.943219+0000","last_fullsized":"2024-03-22T19:09:14.943219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:21.123991+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:09:21.123991+0000","last_peered":"2024-03-22T19:09:21.123991+0000","last_clean":"2024-03-22T19:09:21.123991+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:09:21.123991+0000","last_undegraded":"2024-03-22T19:09:21.123991+0000","last_fullsized":"2024-03-22T19:09:21.123991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643940+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:14.643940+0000","last_peered":"2024-03-22T19:09:14.643940+0000","last_clean":"2024-03-22T19:09:14.643940+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:14.643940+0000","last_undegraded":"2024-03-22T19:09:14.643940+0000","last_fullsized":"2024-03-22T19:09:14.643940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.651978+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:13.651978+0000","last_peered":"2024-03-22T19:09:13.651978+0000","last_clean":"2024-03-22T19:09:13.651978+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:13.651978+0000","last_undegraded":"2024-03-22T19:09:13.651978+0000","last_fullsized":"2024-03-22T19:09:13.651978+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942973+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:14.942973+0000","last_peered":"2024-03-22T19:09:14.942973+0000","last_clean":"2024-03-22T19:09:14.942973+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:14.942973+0000","last_undegraded":"2024-03-22T19:09:14.942973+0000","last_fullsized":"2024-03-22T19:09:14.942973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.147157+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:18.147157+0000","last_peered":"2024-03-22T19:09:18.147157+0000","last_clean":"2024-03-22T19:09:18.147157+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:18.147157+0000","last_undegraded":"2024-03-22T19:09:18.147157+0000","last_fullsized":"2024-03-22T19:09:18.147157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530391+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:16.530391+0000","last_peered":"2024-03-22T19:09:16.530391+0000","last_clean":"2024-03-22T19:09:16.530391+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:16.530391+0000","last_undegraded":"2024-03-22T19:09:16.530391+0000","last_fullsized":"2024-03-22T19:09:16.530391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.165825+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:20.165825+0000","last_peered":"2024-03-22T19:09:20.165825+0000","last_clean":"2024-03-22T19:09:20.165825+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:20.165825+0000","last_undegraded":"2024-03-22T19:09:20.165825+0000","last_fullsized":"2024-03-22T19:09:20.165825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.110990+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:16.110990+0000","last_peered":"2024-03-22T19:09:16.110990+0000","last_clean":"2024-03-22T19:09:16.110990+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:16.110990+0000","last_undegraded":"2024-03-22T19:09:16.110990+0000","last_fullsized":"2024-03-22T19:09:16.110990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.145756+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:09:17.145756+0000","last_peered":"2024-03-22T19:09:17.145756+0000","last_clean":"2024-03-22T19:09:17.145756+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:09:17.145756+0000","last_undegraded":"2024-03-22T19:09:17.145756+0000","last_fullsized":"2024-03-22T19:09:17.145756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.645414+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:13.645414+0000","last_peered":"2024-03-22T19:09:13.645414+0000","last_clean":"2024-03-22T19:09:13.645414+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:13.645414+0000","last_undegraded":"2024-03-22T19:09:13.645414+0000","last_fullsized":"2024-03-22T19:09:13.645414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.164904+0000","last_change":"2024-03-22T19:09:13.141028+0000","last_active":"2024-03-22T19:09:20.164904+0000","last_peered":"2024-03-22T19:09:20.164904+0000","last_clean":"2024-03-22T19:09:20.164904+0000","last_became_active":"2024-03-22T19:09:13.140682+0000","last_became_peered":"2024-03-22T19:09:13.140682+0000","last_unstale":"2024-03-22T19:09:20.164904+0000","last_undegraded":"2024-03-22T19:09:20.164904+0000","last_fullsized":"2024-03-22T19:09:20.164904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:59:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957605+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:14.957605+0000","last_peered":"2024-03-22T19:09:14.957605+0000","last_clean":"2024-03-22T19:09:14.957605+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:14.957605+0000","last_undegraded":"2024-03-22T19:09:14.957605+0000","last_fullsized":"2024-03-22T19:09:14.957605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.155132+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:19.155132+0000","last_peered":"2024-03-22T19:09:19.155132+0000","last_clean":"2024-03-22T19:09:19.155132+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:19.155132+0000","last_undegraded":"2024-03-22T19:09:19.155132+0000","last_fullsized":"2024-03-22T19:09:19.155132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.653290+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:13.653290+0000","last_peered":"2024-03-22T19:09:13.653290+0000","last_clean":"2024-03-22T19:09:13.653290+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:13.653290+0000","last_undegraded":"2024-03-22T19:09:13.653290+0000","last_fullsized":"2024-03-22T19:09:13.653290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.144051+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:09:17.144051+0000","last_peered":"2024-03-22T19:09:17.144051+0000","last_clean":"2024-03-22T19:09:17.144051+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:09:17.144051+0000","last_undegraded":"2024-03-22T19:09:17.144051+0000","last_fullsized":"2024-03-22T19:09:17.144051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643993+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:14.643993+0000","last_peered":"2024-03-22T19:09:14.643993+0000","last_clean":"2024-03-22T19:09:14.643993+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:14.643993+0000","last_undegraded":"2024-03-22T19:09:14.643993+0000","last_fullsized":"2024-03-22T19:09:14.643993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.166226+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:20.166226+0000","last_peered":"2024-03-22T19:09:20.166226+0000","last_clean":"2024-03-22T19:09:20.166226+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:20.166226+0000","last_undegraded":"2024-03-22T19:09:20.166226+0000","last_fullsized":"2024-03-22T19:09:20.166226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527784+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:14.527784+0000","last_peered":"2024-03-22T19:09:14.527784+0000","last_clean":"2024-03-22T19:09:14.527784+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:14.527784+0000","last_undegraded":"2024-03-22T19:09:14.527784+0000","last_fullsized":"2024-03-22T19:09:14.527784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.165154+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:20.165154+0000","last_peered":"2024-03-22T19:09:20.165154+0000","last_clean":"2024-03-22T19:09:20.165154+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:20.165154+0000","last_undegraded":"2024-03-22T19:09:20.165154+0000","last_fullsized":"2024-03-22T19:09:20.165154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631321+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:14.631321+0000","last_peered":"2024-03-22T19:09:14.631321+0000","last_clean":"2024-03-22T19:09:14.631321+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:14.631321+0000","last_undegraded":"2024-03-22T19:09:14.631321+0000","last_fullsized":"2024-03-22T19:09:14.631321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.152419+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:09:19.152419+0000","last_peered":"2024-03-22T19:09:19.152419+0000","last_clean":"2024-03-22T19:09:19.152419+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:09:19.152419+0000","last_undegraded":"2024-03-22T19:09:19.152419+0000","last_fullsized":"2024-03-22T19:09:19.152419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630582+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:14.630582+0000","last_peered":"2024-03-22T19:09:14.630582+0000","last_clean":"2024-03-22T19:09:14.630582+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:14.630582+0000","last_undegraded":"2024-03-22T19:09:14.630582+0000","last_fullsized":"2024-03-22T19:09:14.630582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'16","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.146345+0000","last_change":"2024-03-22T19:09:13.143469+0000","last_active":"2024-03-22T19:09:18.146345+0000","last_peered":"2024-03-22T19:09:18.146345+0000","last_clean":"2024-03-22T19:09:18.146345+0000","last_became_active":"2024-03-22T19:09:13.142918+0000","last_became_peered":"2024-03-22T19:09:13.142918+0000","last_unstale":"2024-03-22T19:09:18.146345+0000","last_undegraded":"2024-03-22T19:09:18.146345+0000","last_fullsized":"2024-03-22T19:09:18.146345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:16:47.530010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942890+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:14.942890+0000","last_peered":"2024-03-22T19:09:14.942890+0000","last_clean":"2024-03-22T19:09:14.942890+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:14.942890+0000","last_undegraded":"2024-03-22T19:09:14.942890+0000","last_fullsized":"2024-03-22T19:09:14.942890+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.644045+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:14.644045+0000","last_peered":"2024-03-22T19:09:14.644045+0000","last_clean":"2024-03-22T19:09:14.644045+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:14.644045+0000","last_undegraded":"2024-03-22T19:09:14.644045+0000","last_fullsized":"2024-03-22T19:09:14.644045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804498+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:15.804498+0000","last_peered":"2024-03-22T19:09:15.804498+0000","last_clean":"2024-03-22T19:09:15.804498+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:15.804498+0000","last_undegraded":"2024-03-22T19:09:15.804498+0000","last_fullsized":"2024-03-22T19:09:15.804498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.141642+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:17.141642+0000","last_peered":"2024-03-22T19:09:17.141642+0000","last_clean":"2024-03-22T19:09:17.141642+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:17.141642+0000","last_undegraded":"2024-03-22T19:09:17.141642+0000","last_fullsized":"2024-03-22T19:09:17.141642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.140782+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:17.140782+0000","last_peered":"2024-03-22T19:09:17.140782+0000","last_clean":"2024-03-22T19:09:17.140782+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:17.140782+0000","last_undegraded":"2024-03-22T19:09:17.140782+0000","last_fullsized":"2024-03-22T19:09:17.140782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:21.121013+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:09:21.121013+0000","last_peered":"2024-03-22T19:09:21.121013+0000","last_clean":"2024-03-22T19:09:21.121013+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:09:21.121013+0000","last_undegraded":"2024-03-22T19:09:21.121013+0000","last_fullsized":"2024-03-22T19:09:21.121013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.140871+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:17.140871+0000","last_peered":"2024-03-22T19:09:17.140871+0000","last_clean":"2024-03-22T19:09:17.140871+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:17.140871+0000","last_undegraded":"2024-03-22T19:09:17.140871+0000","last_fullsized":"2024-03-22T19:09:17.140871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942980+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:14.942980+0000","last_peered":"2024-03-22T19:09:14.942980+0000","last_clean":"2024-03-22T19:09:14.942980+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:14.942980+0000","last_undegraded":"2024-03-22T19:09:14.942980+0000","last_fullsized":"2024-03-22T19:09:14.942980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.147356+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:09:18.147356+0000","last_peered":"2024-03-22T19:09:18.147356+0000","last_clean":"2024-03-22T19:09:18.147356+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:09:18.147356+0000","last_undegraded":"2024-03-22T19:09:18.147356+0000","last_fullsized":"2024-03-22T19:09:18.147356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530442+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:16.530442+0000","last_peered":"2024-03-22T19:09:16.530442+0000","last_clean":"2024-03-22T19:09:16.530442+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:16.530442+0000","last_undegraded":"2024-03-22T19:09:16.530442+0000","last_fullsized":"2024-03-22T19:09:16.530442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.154246+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:19.154246+0000","last_peered":"2024-03-22T19:09:19.154246+0000","last_clean":"2024-03-22T19:09:19.154246+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:19.154246+0000","last_undegraded":"2024-03-22T19:09:19.154246+0000","last_fullsized":"2024-03-22T19:09:19.154246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.627015+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:13.627015+0000","last_peered":"2024-03-22T19:09:13.627015+0000","last_clean":"2024-03-22T19:09:13.627015+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:13.627015+0000","last_undegraded":"2024-03-22T19:09:13.627015+0000","last_fullsized":"2024-03-22T19:09:13.627015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527730+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:14.527730+0000","last_peered":"2024-03-22T19:09:14.527730+0000","last_clean":"2024-03-22T19:09:14.527730+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:14.527730+0000","last_undegraded":"2024-03-22T19:09:14.527730+0000","last_fullsized":"2024-03-22T19:09:14.527730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.145449+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:18.145449+0000","last_peered":"2024-03-22T19:09:18.145449+0000","last_clean":"2024-03-22T19:09:18.145449+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:18.145449+0000","last_undegraded":"2024-03-22T19:09:18.145449+0000","last_fullsized":"2024-03-22T19:09:18.145449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.155143+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:19.155143+0000","last_peered":"2024-03-22T19:09:19.155143+0000","last_clean":"2024-03-22T19:09:19.155143+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:19.155143+0000","last_undegraded":"2024-03-22T19:09:19.155143+0000","last_fullsized":"2024-03-22T19:09:19.155143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527822+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:14.527822+0000","last_peered":"2024-03-22T19:09:14.527822+0000","last_clean":"2024-03-22T19:09:14.527822+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:14.527822+0000","last_undegraded":"2024-03-22T19:09:14.527822+0000","last_fullsized":"2024-03-22T19:09:14.527822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.154825+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:19.154825+0000","last_peered":"2024-03-22T19:09:19.154825+0000","last_clean":"2024-03-22T19:09:19.154825+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:19.154825+0000","last_undegraded":"2024-03-22T19:09:19.154825+0000","last_fullsized":"2024-03-22T19:09:19.154825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530538+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:16.530538+0000","last_peered":"2024-03-22T19:09:16.530538+0000","last_clean":"2024-03-22T19:09:16.530538+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:16.530538+0000","last_undegraded":"2024-03-22T19:09:16.530538+0000","last_fullsized":"2024-03-22T19:09:16.530538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.141605+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:17.141605+0000","last_peered":"2024-03-22T19:09:17.141605+0000","last_clean":"2024-03-22T19:09:17.141605+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:17.141605+0000","last_undegraded":"2024-03-22T19:09:17.141605+0000","last_fullsized":"2024-03-22T19:09:17.141605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957545+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:14.957545+0000","last_peered":"2024-03-22T19:09:14.957545+0000","last_clean":"2024-03-22T19:09:14.957545+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:14.957545+0000","last_undegraded":"2024-03-22T19:09:14.957545+0000","last_fullsized":"2024-03-22T19:09:14.957545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.142313+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:17.142313+0000","last_peered":"2024-03-22T19:09:17.142313+0000","last_clean":"2024-03-22T19:09:17.142313+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:17.142313+0000","last_undegraded":"2024-03-22T19:09:17.142313+0000","last_fullsized":"2024-03-22T19:09:17.142313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643928+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:14.643928+0000","last_peered":"2024-03-22T19:09:14.643928+0000","last_clean":"2024-03-22T19:09:14.643928+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:14.643928+0000","last_undegraded":"2024-03-22T19:09:14.643928+0000","last_fullsized":"2024-03-22T19:09:14.643928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.166485+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:20.166485+0000","last_peered":"2024-03-22T19:09:20.166485+0000","last_clean":"2024-03-22T19:09:20.166485+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:20.166485+0000","last_undegraded":"2024-03-22T19:09:20.166485+0000","last_fullsized":"2024-03-22T19:09:20.166485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804419+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:15.804419+0000","last_peered":"2024-03-22T19:09:15.804419+0000","last_clean":"2024-03-22T19:09:15.804419+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:15.804419+0000","last_undegraded":"2024-03-22T19:09:15.804419+0000","last_fullsized":"2024-03-22T19:09:15.804419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.153605+0000","last_change":"2024-03-22T19:09:13.144416+0000","last_active":"2024-03-22T19:09:19.153605+0000","last_peered":"2024-03-22T19:09:19.153605+0000","last_clean":"2024-03-22T19:09:19.153605+0000","last_became_active":"2024-03-22T19:09:13.144063+0000","last_became_peered":"2024-03-22T19:09:13.144063+0000","last_unstale":"2024-03-22T19:09:19.153605+0000","last_undegraded":"2024-03-22T19:09:19.153605+0000","last_fullsized":"2024-03-22T19:09:19.153605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:56:53.881418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530451+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:16.530451+0000","last_peered":"2024-03-22T19:09:16.530451+0000","last_clean":"2024-03-22T19:09:16.530451+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:16.530451+0000","last_undegraded":"2024-03-22T19:09:16.530451+0000","last_fullsized":"2024-03-22T19:09:16.530451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.143494+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:17.143494+0000","last_peered":"2024-03-22T19:09:17.143494+0000","last_clean":"2024-03-22T19:09:17.143494+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:17.143494+0000","last_undegraded":"2024-03-22T19:09:17.143494+0000","last_fullsized":"2024-03-22T19:09:17.143494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630990+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:14.630990+0000","last_peered":"2024-03-22T19:09:14.630990+0000","last_clean":"2024-03-22T19:09:14.630990+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:14.630990+0000","last_undegraded":"2024-03-22T19:09:14.630990+0000","last_fullsized":"2024-03-22T19:09:14.630990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.153279+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:09:19.153279+0000","last_peered":"2024-03-22T19:09:19.153279+0000","last_clean":"2024-03-22T19:09:19.153279+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:19.153279+0000","last_undegraded":"2024-03-22T19:09:19.153279+0000","last_fullsized":"2024-03-22T19:09:19.153279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631182+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:14.631182+0000","last_peered":"2024-03-22T19:09:14.631182+0000","last_clean":"2024-03-22T19:09:14.631182+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:14.631182+0000","last_undegraded":"2024-03-22T19:09:14.631182+0000","last_fullsized":"2024-03-22T19:09:14.631182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.165201+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:20.165201+0000","last_peered":"2024-03-22T19:09:20.165201+0000","last_clean":"2024-03-22T19:09:20.165201+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:20.165201+0000","last_undegraded":"2024-03-22T19:09:20.165201+0000","last_fullsized":"2024-03-22T19:09:20.165201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.599826+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:16.599826+0000","last_peered":"2024-03-22T19:09:16.599826+0000","last_clean":"2024-03-22T19:09:16.599826+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:16.599826+0000","last_undegraded":"2024-03-22T19:09:16.599826+0000","last_fullsized":"2024-03-22T19:09:16.599826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630530+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:14.630530+0000","last_peered":"2024-03-22T19:09:14.630530+0000","last_clean":"2024-03-22T19:09:14.630530+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:14.630530+0000","last_undegraded":"2024-03-22T19:09:14.630530+0000","last_fullsized":"2024-03-22T19:09:14.630530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631280+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:14.631280+0000","last_peered":"2024-03-22T19:09:14.631280+0000","last_clean":"2024-03-22T19:09:14.631280+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:14.631280+0000","last_undegraded":"2024-03-22T19:09:14.631280+0000","last_fullsized":"2024-03-22T19:09:14.631280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804448+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:15.804448+0000","last_peered":"2024-03-22T19:09:15.804448+0000","last_clean":"2024-03-22T19:09:15.804448+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:15.804448+0000","last_undegraded":"2024-03-22T19:09:15.804448+0000","last_fullsized":"2024-03-22T19:09:15.804448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924566+0000","last_change":"2024-03-22T19:09:13.650295+0000","last_active":"2024-03-22T19:09:14.924566+0000","last_peered":"2024-03-22T19:09:14.924566+0000","last_clean":"2024-03-22T19:09:14.924566+0000","last_became_active":"2024-03-22T19:09:13.649889+0000","last_became_peered":"2024-03-22T19:09:13.649889+0000","last_unstale":"2024-03-22T19:09:14.924566+0000","last_undegraded":"2024-03-22T19:09:14.924566+0000","last_fullsized":"2024-03-22T19:09:14.924566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:25:15.931309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924757+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:14.924757+0000","last_peered":"2024-03-22T19:09:14.924757+0000","last_clean":"2024-03-22T19:09:14.924757+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:14.924757+0000","last_undegraded":"2024-03-22T19:09:14.924757+0000","last_fullsized":"2024-03-22T19:09:14.924757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957320+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:14.957320+0000","last_peered":"2024-03-22T19:09:14.957320+0000","last_clean":"2024-03-22T19:09:14.957320+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:14.957320+0000","last_undegraded":"2024-03-22T19:09:14.957320+0000","last_fullsized":"2024-03-22T19:09:14.957320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527838+0000","last_change":"2024-03-22T19:09:13.648683+0000","last_active":"2024-03-22T19:09:14.527838+0000","last_peered":"2024-03-22T19:09:14.527838+0000","last_clean":"2024-03-22T19:09:14.527838+0000","last_became_active":"2024-03-22T19:09:13.648300+0000","last_became_peered":"2024-03-22T19:09:13.648300+0000","last_unstale":"2024-03-22T19:09:14.527838+0000","last_undegraded":"2024-03-22T19:09:14.527838+0000","last_fullsized":"2024-03-22T19:09:14.527838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:24:31.803987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804480+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:15.804480+0000","last_peered":"2024-03-22T19:09:15.804480+0000","last_clean":"2024-03-22T19:09:15.804480+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:15.804480+0000","last_undegraded":"2024-03-22T19:09:15.804480+0000","last_fullsized":"2024-03-22T19:09:15.804480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924950+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:14.924950+0000","last_peered":"2024-03-22T19:09:14.924950+0000","last_clean":"2024-03-22T19:09:14.924950+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:14.924950+0000","last_undegraded":"2024-03-22T19:09:14.924950+0000","last_fullsized":"2024-03-22T19:09:14.924950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530519+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:16.530519+0000","last_peered":"2024-03-22T19:09:16.530519+0000","last_clean":"2024-03-22T19:09:16.530519+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:16.530519+0000","last_undegraded":"2024-03-22T19:09:16.530519+0000","last_fullsized":"2024-03-22T19:09:16.530519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942745+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:14.942745+0000","last_peered":"2024-03-22T19:09:14.942745+0000","last_clean":"2024-03-22T19:09:14.942745+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:14.942745+0000","last_undegraded":"2024-03-22T19:09:14.942745+0000","last_fullsized":"2024-03-22T19:09:14.942745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530420+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:16.530420+0000","last_peered":"2024-03-22T19:09:16.530420+0000","last_clean":"2024-03-22T19:09:16.530420+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:16.530420+0000","last_undegraded":"2024-03-22T19:09:16.530420+0000","last_fullsized":"2024-03-22T19:09:16.530420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.647757+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:13.647757+0000","last_peered":"2024-03-22T19:09:13.647757+0000","last_clean":"2024-03-22T19:09:13.647757+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:13.647757+0000","last_undegraded":"2024-03-22T19:09:13.647757+0000","last_fullsized":"2024-03-22T19:09:13.647757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527795+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:14.527795+0000","last_peered":"2024-03-22T19:09:14.527795+0000","last_clean":"2024-03-22T19:09:14.527795+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:14.527795+0000","last_undegraded":"2024-03-22T19:09:14.527795+0000","last_fullsized":"2024-03-22T19:09:14.527795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804462+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:15.804462+0000","last_peered":"2024-03-22T19:09:15.804462+0000","last_clean":"2024-03-22T19:09:15.804462+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:15.804462+0000","last_undegraded":"2024-03-22T19:09:15.804462+0000","last_fullsized":"2024-03-22T19:09:15.804462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7368,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":169,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":40048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":413367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776141,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":1004780,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939666,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":413345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.63},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.88800000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.748},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.85399999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168599,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":412630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.439}]},{"osd":1,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.627}]},{"osd":3,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.751}]},{"osd":5,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.5},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.74099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397530,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":1005817,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":412313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72099999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.443}]},{"osd":1,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81699999999999995,"5min":0.875,"15min":0.875},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.369,"15min":1.369},"last":0.746}]},{"osd":2,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.76200000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.53400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71399999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78900000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.749,"15min":0.76100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84599999999999997,"5min":1.55,"15min":1.55},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85799999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Mar 22 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.75900000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.81599999999999995}]},{"osd":9,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.63100000000000001}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":413478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.42599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72599999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.71799999999999997,"15min":0.79400000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":6.71,"15min":6.71},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Mar 22 19:08:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Mar 22 19:08:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.26600000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.375},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:08:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40699999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68100000000000005}]},{"osd":9,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.628}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738419,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":414353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:08:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:08:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45800000000000002,"15min":0.44400000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.372}]},{"osd":4,"last update":"Fri Mar 22 19:08:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:08:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96099999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:08:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.60999999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":421656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Mar 22 19:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.502},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Mar 22 19:08:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.51500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:08:43 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.44700000000000001}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60499999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215144,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":412770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78000000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.42699999999999999},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.024,"5min":1.244,"15min":1.244},"last":0.623},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:08:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1359999999999999,"5min":1.367,"15min":1.367},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0940000000000001,"5min":1.208,"15min":1.208},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Mar 22 19:08:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69899999999999995,"15min":0.70399999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93700000000000006,"5min":1.417,"15min":1.417},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.59699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1080000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Mar 22 19:09:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.077,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94299999999999995,"5min":1.641,"15min":1.641},"last":0.45800000000000002}]},{"osd":7,"last update":"Fri Mar 22 19:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.74399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953509,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":412819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38,"15min":0.377},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:08:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:09:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.628},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.92400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:08:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73699999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.90400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":5,"up_from":37,"seq":158913789986,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1005541,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.71299999999999997,"15min":0.73899999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82399999999999995,"5min":3.887,"15min":3.887},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.49199999999999999}]},{"osd":1,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36599999999999999},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.76300000000000001,"15min":0.78700000000000003},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83999999999999997,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69299999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.86699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:09:23.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T19:09:23.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 -- 172.21.15.86:0/3729825881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 msgr2=0x7f6b9810afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/3729825881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b9810afd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000b0a0 tx=0x7f6b9000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 -- 172.21.15.86:0/3729825881 shutdown_connections 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/3729825881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b9810afd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/3729825881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b980731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/3729825881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b98073790 0x7f6b98073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 -- 172.21.15.86:0/3729825881 >> 172.21.15.86:0/3729825881 conn(0x7f6b980fc1c0 msgr2=0x7f6b980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 -- 172.21.15.86:0/3729825881 shutdown_connections 2024-03-22T19:09:23.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.251+0000 7f6b9fc00700 1 -- 172.21.15.86:0/3729825881 wait complete. 2024-03-22T19:09:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 Processor -- start 2024-03-22T19:09:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 -- start start 2024-03-22T19:09:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b98073790 0x7f6b98198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b981993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b981a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981a09c0 con 0x7f6b98108b40 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981a0b00 con 0x7f6b98073790 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.252+0000 7f6b9fc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981a0c90 con 0x7f6b98074d70 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b981a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b97e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b981993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b97e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b981993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:33600/0 (socket says 172.21.15.86:33600) 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b981a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43074/0 (socket says 172.21.15.86:43074) 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b97e00700 1 -- 172.21.15.86:0/2743544995 learned_addr learned my addr 172.21.15.86:0/2743544995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9d200700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b98073790 0x7f6b98198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 -- 172.21.15.86:0/2743544995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 msgr2=0x7f6b981993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b981993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 -- 172.21.15.86:0/2743544995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b98073790 msgr2=0x7f6b98198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b98073790 0x7f6b98198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 -- 172.21.15.86:0/2743544995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b90009d00 con 0x7f6b98108b40 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b97e00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b981993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.253+0000 7f6b9dc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b981a03f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007b90 tx=0x7f6b90007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.254+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000e040 con 0x7f6b98108b40 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.254+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9000be40 con 0x7f6b98108b40 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.254+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b981a0f10 con 0x7f6b98108b40 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.254+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9001b460 con 0x7f6b98108b40 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.254+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b981a1280 con 0x7f6b98108b40 2024-03-22T19:09:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.255+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b98107950 con 0x7f6b98108b40 2024-03-22T19:09:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.257+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f6b9001d470 con 0x7f6b98108b40 2024-03-22T19:09:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.258+0000 7f6b95600700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6b800a0b20 0x7f6b800a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.258+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6b900cd890 con 0x7f6b98108b40 2024-03-22T19:09:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.258+0000 7f6b9d200700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6b800a0b20 0x7f6b800a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.259+0000 7f6b9d200700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6b800a0b20 0x7f6b800a2fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009c60 tx=0x7f6b8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.260+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6b90097240 con 0x7f6b98108b40 2024-03-22T19:09:23.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.465+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b980611f0 con 0x7f6b800a0b20 2024-03-22T19:09:23.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:23 smithi131 ceph-mon[78559]: pgmap v236: 97 pgs: 97 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 960 B/s rd, 6.2 KiB/s wr, 21 op/s 2024-03-22T19:09:23.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:23 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3725279498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:23.472 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:09:23 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: 2024-03-22T19:09:23.124+0000 7f4cb7fe8700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-22T19:09:23.472 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:09:23 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: 2024-03-22T19:09:23.124+0000 7f4cb7fe8700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-22T19:09:23.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.474+0000 7f6b95600700 1 -- 172.21.15.86:0/2743544995 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+331787 (secure 0 0 0) 0x7f6b980611f0 con 0x7f6b800a0b20 2024-03-22T19:09:23.480 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6b800a0b20 msgr2=0x7f6b800a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6b800a0b20 0x7f6b800a2fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009c60 tx=0x7f6b8c009260 comp rx=0 tx=0).stop 2024-03-22T19:09:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 msgr2=0x7f6b981a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b981a03f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007b90 tx=0x7f6b90007930 comp rx=0 tx=0).stop 2024-03-22T19:09:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 shutdown_connections 2024-03-22T19:09:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6b800a0b20 0x7f6b800a2fe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b98108b40 0x7f6b981a03f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b98073790 0x7f6b98198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 --2- 172.21.15.86:0/2743544995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b98074d70 0x7f6b981993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 >> 172.21.15.86:0/2743544995 conn(0x7f6b980fc1c0 msgr2=0x7f6b98104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 shutdown_connections 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:23.479+0000 7f6b9fc00700 1 -- 172.21.15.86:0/2743544995 wait complete. 2024-03-22T19:09:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:09:23.497 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:23 smithi086 ceph-mon[38032]: pgmap v236: 97 pgs: 97 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 960 B/s rd, 6.2 KiB/s wr, 21 op/s 2024-03-22T19:09:23.497 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:23 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3725279498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:23.499 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-03-22T19:09:22.150841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597853,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":136,"num_read_kb":117,"num_write":351,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334688,"kb_used_data":10484,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124607136,"statfs":{"total":1151940427776,"available":1151597707264,"internally_reserved":0,"allocated":10735616,"data_stored":6742869,"data_compressed":106959,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7485,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":170,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.529368"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804515+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:15.804515+0000","last_peered":"2024-03-22T19:09:15.804515+0000","last_clean":"2024-03-22T19:09:15.804515+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:15.804515+0000","last_undegraded":"2024-03-22T19:09:15.804515+0000","last_fullsized":"2024-03-22T19:09:15.804515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957320+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:14.957320+0000","last_peered":"2024-03-22T19:09:14.957320+0000","last_clean":"2024-03-22T19:09:14.957320+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:14.957320+0000","last_undegraded":"2024-03-22T19:09:14.957320+0000","last_fullsized":"2024-03-22T19:09:14.957320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957385+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:14.957385+0000","last_peered":"2024-03-22T19:09:14.957385+0000","last_clean":"2024-03-22T19:09:14.957385+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:14.957385+0000","last_undegraded":"2024-03-22T19:09:14.957385+0000","last_fullsized":"2024-03-22T19:09:14.957385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.093072+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:15.093072+0000","last_peered":"2024-03-22T19:09:15.093072+0000","last_clean":"2024-03-22T19:09:15.093072+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:15.093072+0000","last_undegraded":"2024-03-22T19:09:15.093072+0000","last_fullsized":"2024-03-22T19:09:15.093072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643954+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:14.643954+0000","last_peered":"2024-03-22T19:09:14.643954+0000","last_clean":"2024-03-22T19:09:14.643954+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:14.643954+0000","last_undegraded":"2024-03-22T19:09:14.643954+0000","last_fullsized":"2024-03-22T19:09:14.643954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.140787+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:17.140787+0000","last_peered":"2024-03-22T19:09:17.140787+0000","last_clean":"2024-03-22T19:09:17.140787+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:17.140787+0000","last_undegraded":"2024-03-22T19:09:17.140787+0000","last_fullsized":"2024-03-22T19:09:17.140787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630682+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:14.630682+0000","last_peered":"2024-03-22T19:09:14.630682+0000","last_clean":"2024-03-22T19:09:14.630682+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:14.630682+0000","last_undegraded":"2024-03-22T19:09:14.630682+0000","last_fullsized":"2024-03-22T19:09:14.630682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530409+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:16.530409+0000","last_peered":"2024-03-22T19:09:16.530409+0000","last_clean":"2024-03-22T19:09:16.530409+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:16.530409+0000","last_undegraded":"2024-03-22T19:09:16.530409+0000","last_fullsized":"2024-03-22T19:09:16.530409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.644746+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:13.644746+0000","last_peered":"2024-03-22T19:09:13.644746+0000","last_clean":"2024-03-22T19:09:13.644746+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:13.644746+0000","last_undegraded":"2024-03-22T19:09:13.644746+0000","last_fullsized":"2024-03-22T19:09:13.644746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.141158+0000","last_change":"2024-03-22T19:09:13.651761+0000","last_active":"2024-03-22T19:09:17.141158+0000","last_peered":"2024-03-22T19:09:17.141158+0000","last_clean":"2024-03-22T19:09:17.141158+0000","last_became_active":"2024-03-22T19:09:13.651374+0000","last_became_peered":"2024-03-22T19:09:13.651374+0000","last_unstale":"2024-03-22T19:09:17.141158+0000","last_undegraded":"2024-03-22T19:09:17.141158+0000","last_fullsized":"2024-03-22T19:09:17.141158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:08:41.358335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631127+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:14.631127+0000","last_peered":"2024-03-22T19:09:14.631127+0000","last_clean":"2024-03-22T19:09:14.631127+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:14.631127+0000","last_undegraded":"2024-03-22T19:09:14.631127+0000","last_fullsized":"2024-03-22T19:09:14.631127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.093061+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:15.093061+0000","last_peered":"2024-03-22T19:09:15.093061+0000","last_clean":"2024-03-22T19:09:15.093061+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:15.093061+0000","last_undegraded":"2024-03-22T19:09:15.093061+0000","last_fullsized":"2024-03-22T19:09:15.093061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527806+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:14.527806+0000","last_peered":"2024-03-22T19:09:14.527806+0000","last_clean":"2024-03-22T19:09:14.527806+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:14.527806+0000","last_undegraded":"2024-03-22T19:09:14.527806+0000","last_fullsized":"2024-03-22T19:09:14.527806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957204+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:14.957204+0000","last_peered":"2024-03-22T19:09:14.957204+0000","last_clean":"2024-03-22T19:09:14.957204+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:14.957204+0000","last_undegraded":"2024-03-22T19:09:14.957204+0000","last_fullsized":"2024-03-22T19:09:14.957204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957125+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:14.957125+0000","last_peered":"2024-03-22T19:09:14.957125+0000","last_clean":"2024-03-22T19:09:14.957125+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:14.957125+0000","last_undegraded":"2024-03-22T19:09:14.957125+0000","last_fullsized":"2024-03-22T19:09:14.957125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.645969+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:13.645969+0000","last_peered":"2024-03-22T19:09:13.645969+0000","last_clean":"2024-03-22T19:09:13.645969+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:13.645969+0000","last_undegraded":"2024-03-22T19:09:13.645969+0000","last_fullsized":"2024-03-22T19:09:13.645969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804526+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:15.804526+0000","last_peered":"2024-03-22T19:09:15.804526+0000","last_clean":"2024-03-22T19:09:15.804526+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:15.804526+0000","last_undegraded":"2024-03-22T19:09:15.804526+0000","last_fullsized":"2024-03-22T19:09:15.804526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.111027+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:16.111027+0000","last_peered":"2024-03-22T19:09:16.111027+0000","last_clean":"2024-03-22T19:09:16.111027+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:16.111027+0000","last_undegraded":"2024-03-22T19:09:16.111027+0000","last_fullsized":"2024-03-22T19:09:16.111027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924829+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:14.924829+0000","last_peered":"2024-03-22T19:09:14.924829+0000","last_clean":"2024-03-22T19:09:14.924829+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:14.924829+0000","last_undegraded":"2024-03-22T19:09:14.924829+0000","last_fullsized":"2024-03-22T19:09:14.924829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.166749+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:09:20.166749+0000","last_peered":"2024-03-22T19:09:20.166749+0000","last_clean":"2024-03-22T19:09:20.166749+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:09:20.166749+0000","last_undegraded":"2024-03-22T19:09:20.166749+0000","last_fullsized":"2024-03-22T19:09:20.166749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530460+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:16.530460+0000","last_peered":"2024-03-22T19:09:16.530460+0000","last_clean":"2024-03-22T19:09:16.530460+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:16.530460+0000","last_undegraded":"2024-03-22T19:09:16.530460+0000","last_fullsized":"2024-03-22T19:09:16.530460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.144432+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:17.144432+0000","last_peered":"2024-03-22T19:09:17.144432+0000","last_clean":"2024-03-22T19:09:17.144432+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:17.144432+0000","last_undegraded":"2024-03-22T19:09:17.144432+0000","last_fullsized":"2024-03-22T19:09:17.144432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957612+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:14.957612+0000","last_peered":"2024-03-22T19:09:14.957612+0000","last_clean":"2024-03-22T19:09:14.957612+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:14.957612+0000","last_undegraded":"2024-03-22T19:09:14.957612+0000","last_fullsized":"2024-03-22T19:09:14.957612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.153511+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:19.153511+0000","last_peered":"2024-03-22T19:09:19.153511+0000","last_clean":"2024-03-22T19:09:19.153511+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:19.153511+0000","last_undegraded":"2024-03-22T19:09:19.153511+0000","last_fullsized":"2024-03-22T19:09:19.153511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.647575+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:13.647575+0000","last_peered":"2024-03-22T19:09:13.647575+0000","last_clean":"2024-03-22T19:09:13.647575+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:13.647575+0000","last_undegraded":"2024-03-22T19:09:13.647575+0000","last_fullsized":"2024-03-22T19:09:13.647575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643981+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:14.643981+0000","last_peered":"2024-03-22T19:09:14.643981+0000","last_clean":"2024-03-22T19:09:14.643981+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:14.643981+0000","last_undegraded":"2024-03-22T19:09:14.643981+0000","last_fullsized":"2024-03-22T19:09:14.643981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.943219+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:14.943219+0000","last_peered":"2024-03-22T19:09:14.943219+0000","last_clean":"2024-03-22T19:09:14.943219+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:14.943219+0000","last_undegraded":"2024-03-22T19:09:14.943219+0000","last_fullsized":"2024-03-22T19:09:14.943219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:21.123991+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:09:21.123991+0000","last_peered":"2024-03-22T19:09:21.123991+0000","last_clean":"2024-03-22T19:09:21.123991+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:09:21.123991+0000","last_undegraded":"2024-03-22T19:09:21.123991+0000","last_fullsized":"2024-03-22T19:09:21.123991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643940+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:14.643940+0000","last_peered":"2024-03-22T19:09:14.643940+0000","last_clean":"2024-03-22T19:09:14.643940+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:14.643940+0000","last_undegraded":"2024-03-22T19:09:14.643940+0000","last_fullsized":"2024-03-22T19:09:14.643940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.651978+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:13.651978+0000","last_peered":"2024-03-22T19:09:13.651978+0000","last_clean":"2024-03-22T19:09:13.651978+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:13.651978+0000","last_undegraded":"2024-03-22T19:09:13.651978+0000","last_fullsized":"2024-03-22T19:09:13.651978+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942973+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:14.942973+0000","last_peered":"2024-03-22T19:09:14.942973+0000","last_clean":"2024-03-22T19:09:14.942973+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:14.942973+0000","last_undegraded":"2024-03-22T19:09:14.942973+0000","last_fullsized":"2024-03-22T19:09:14.942973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.147157+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:18.147157+0000","last_peered":"2024-03-22T19:09:18.147157+0000","last_clean":"2024-03-22T19:09:18.147157+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:18.147157+0000","last_undegraded":"2024-03-22T19:09:18.147157+0000","last_fullsized":"2024-03-22T19:09:18.147157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530391+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:16.530391+0000","last_peered":"2024-03-22T19:09:16.530391+0000","last_clean":"2024-03-22T19:09:16.530391+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:16.530391+0000","last_undegraded":"2024-03-22T19:09:16.530391+0000","last_fullsized":"2024-03-22T19:09:16.530391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.165825+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:20.165825+0000","last_peered":"2024-03-22T19:09:20.165825+0000","last_clean":"2024-03-22T19:09:20.165825+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:20.165825+0000","last_undegraded":"2024-03-22T19:09:20.165825+0000","last_fullsized":"2024-03-22T19:09:20.165825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.110990+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:16.110990+0000","last_peered":"2024-03-22T19:09:16.110990+0000","last_clean":"2024-03-22T19:09:16.110990+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:16.110990+0000","last_undegraded":"2024-03-22T19:09:16.110990+0000","last_fullsized":"2024-03-22T19:09:16.110990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.145756+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:09:17.145756+0000","last_peered":"2024-03-22T19:09:17.145756+0000","last_clean":"2024-03-22T19:09:17.145756+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:09:17.145756+0000","last_undegraded":"2024-03-22T19:09:17.145756+0000","last_fullsized":"2024-03-22T19:09:17.145756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.645414+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:13.645414+0000","last_peered":"2024-03-22T19:09:13.645414+0000","last_clean":"2024-03-22T19:09:13.645414+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:13.645414+0000","last_undegraded":"2024-03-22T19:09:13.645414+0000","last_fullsized":"2024-03-22T19:09:13.645414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.164904+0000","last_change":"2024-03-22T19:09:13.141028+0000","last_active":"2024-03-22T19:09:20.164904+0000","last_peered":"2024-03-22T19:09:20.164904+0000","last_clean":"2024-03-22T19:09:20.164904+0000","last_became_active":"2024-03-22T19:09:13.140682+0000","last_became_peered":"2024-03-22T19:09:13.140682+0000","last_unstale":"2024-03-22T19:09:20.164904+0000","last_undegraded":"2024-03-22T19:09:20.164904+0000","last_fullsized":"2024-03-22T19:09:20.164904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:59:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957605+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:14.957605+0000","last_peered":"2024-03-22T19:09:14.957605+0000","last_clean":"2024-03-22T19:09:14.957605+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:14.957605+0000","last_undegraded":"2024-03-22T19:09:14.957605+0000","last_fullsized":"2024-03-22T19:09:14.957605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.155132+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:19.155132+0000","last_peered":"2024-03-22T19:09:19.155132+0000","last_clean":"2024-03-22T19:09:19.155132+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:19.155132+0000","last_undegraded":"2024-03-22T19:09:19.155132+0000","last_fullsized":"2024-03-22T19:09:19.155132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.653290+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:13.653290+0000","last_peered":"2024-03-22T19:09:13.653290+0000","last_clean":"2024-03-22T19:09:13.653290+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:13.653290+0000","last_undegraded":"2024-03-22T19:09:13.653290+0000","last_fullsized":"2024-03-22T19:09:13.653290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.144051+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:09:17.144051+0000","last_peered":"2024-03-22T19:09:17.144051+0000","last_clean":"2024-03-22T19:09:17.144051+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:09:17.144051+0000","last_undegraded":"2024-03-22T19:09:17.144051+0000","last_fullsized":"2024-03-22T19:09:17.144051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643993+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:14.643993+0000","last_peered":"2024-03-22T19:09:14.643993+0000","last_clean":"2024-03-22T19:09:14.643993+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:14.643993+0000","last_undegraded":"2024-03-22T19:09:14.643993+0000","last_fullsized":"2024-03-22T19:09:14.643993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.166226+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:20.166226+0000","last_peered":"2024-03-22T19:09:20.166226+0000","last_clean":"2024-03-22T19:09:20.166226+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:20.166226+0000","last_undegraded":"2024-03-22T19:09:20.166226+0000","last_fullsized":"2024-03-22T19:09:20.166226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527784+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:14.527784+0000","last_peered":"2024-03-22T19:09:14.527784+0000","last_clean":"2024-03-22T19:09:14.527784+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:14.527784+0000","last_undegraded":"2024-03-22T19:09:14.527784+0000","last_fullsized":"2024-03-22T19:09:14.527784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.165154+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:20.165154+0000","last_peered":"2024-03-22T19:09:20.165154+0000","last_clean":"2024-03-22T19:09:20.165154+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:20.165154+0000","last_undegraded":"2024-03-22T19:09:20.165154+0000","last_fullsized":"2024-03-22T19:09:20.165154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631321+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:14.631321+0000","last_peered":"2024-03-22T19:09:14.631321+0000","last_clean":"2024-03-22T19:09:14.631321+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:14.631321+0000","last_undegraded":"2024-03-22T19:09:14.631321+0000","last_fullsized":"2024-03-22T19:09:14.631321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.152419+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:09:19.152419+0000","last_peered":"2024-03-22T19:09:19.152419+0000","last_clean":"2024-03-22T19:09:19.152419+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:09:19.152419+0000","last_undegraded":"2024-03-22T19:09:19.152419+0000","last_fullsized":"2024-03-22T19:09:19.152419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630582+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:14.630582+0000","last_peered":"2024-03-22T19:09:14.630582+0000","last_clean":"2024-03-22T19:09:14.630582+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:14.630582+0000","last_undegraded":"2024-03-22T19:09:14.630582+0000","last_fullsized":"2024-03-22T19:09:14.630582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'16","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.146345+0000","last_change":"2024-03-22T19:09:13.143469+0000","last_active":"2024-03-22T19:09:18.146345+0000","last_peered":"2024-03-22T19:09:18.146345+0000","last_clean":"2024-03-22T19:09:18.146345+0000","last_became_active":"2024-03-22T19:09:13.142918+0000","last_became_peered":"2024-03-22T19:09:13.142918+0000","last_unstale":"2024-03-22T19:09:18.146345+0000","last_undegraded":"2024-03-22T19:09:18.146345+0000","last_fullsized":"2024-03-22T19:09:18.146345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:16:47.530010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942890+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:14.942890+0000","last_peered":"2024-03-22T19:09:14.942890+0000","last_clean":"2024-03-22T19:09:14.942890+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:14.942890+0000","last_undegraded":"2024-03-22T19:09:14.942890+0000","last_fullsized":"2024-03-22T19:09:14.942890+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.644045+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:14.644045+0000","last_peered":"2024-03-22T19:09:14.644045+0000","last_clean":"2024-03-22T19:09:14.644045+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:14.644045+0000","last_undegraded":"2024-03-22T19:09:14.644045+0000","last_fullsized":"2024-03-22T19:09:14.644045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804498+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:15.804498+0000","last_peered":"2024-03-22T19:09:15.804498+0000","last_clean":"2024-03-22T19:09:15.804498+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:15.804498+0000","last_undegraded":"2024-03-22T19:09:15.804498+0000","last_fullsized":"2024-03-22T19:09:15.804498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.141642+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:17.141642+0000","last_peered":"2024-03-22T19:09:17.141642+0000","last_clean":"2024-03-22T19:09:17.141642+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:17.141642+0000","last_undegraded":"2024-03-22T19:09:17.141642+0000","last_fullsized":"2024-03-22T19:09:17.141642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.140782+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:17.140782+0000","last_peered":"2024-03-22T19:09:17.140782+0000","last_clean":"2024-03-22T19:09:17.140782+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:17.140782+0000","last_undegraded":"2024-03-22T19:09:17.140782+0000","last_fullsized":"2024-03-22T19:09:17.140782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:21.121013+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:09:21.121013+0000","last_peered":"2024-03-22T19:09:21.121013+0000","last_clean":"2024-03-22T19:09:21.121013+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:09:21.121013+0000","last_undegraded":"2024-03-22T19:09:21.121013+0000","last_fullsized":"2024-03-22T19:09:21.121013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.140871+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:17.140871+0000","last_peered":"2024-03-22T19:09:17.140871+0000","last_clean":"2024-03-22T19:09:17.140871+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:17.140871+0000","last_undegraded":"2024-03-22T19:09:17.140871+0000","last_fullsized":"2024-03-22T19:09:17.140871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942980+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:14.942980+0000","last_peered":"2024-03-22T19:09:14.942980+0000","last_clean":"2024-03-22T19:09:14.942980+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:14.942980+0000","last_undegraded":"2024-03-22T19:09:14.942980+0000","last_fullsized":"2024-03-22T19:09:14.942980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.147356+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:09:18.147356+0000","last_peered":"2024-03-22T19:09:18.147356+0000","last_clean":"2024-03-22T19:09:18.147356+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:09:18.147356+0000","last_undegraded":"2024-03-22T19:09:18.147356+0000","last_fullsized":"2024-03-22T19:09:18.147356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530442+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:16.530442+0000","last_peered":"2024-03-22T19:09:16.530442+0000","last_clean":"2024-03-22T19:09:16.530442+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:16.530442+0000","last_undegraded":"2024-03-22T19:09:16.530442+0000","last_fullsized":"2024-03-22T19:09:16.530442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.154246+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:19.154246+0000","last_peered":"2024-03-22T19:09:19.154246+0000","last_clean":"2024-03-22T19:09:19.154246+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:19.154246+0000","last_undegraded":"2024-03-22T19:09:19.154246+0000","last_fullsized":"2024-03-22T19:09:19.154246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.627015+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:13.627015+0000","last_peered":"2024-03-22T19:09:13.627015+0000","last_clean":"2024-03-22T19:09:13.627015+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:13.627015+0000","last_undegraded":"2024-03-22T19:09:13.627015+0000","last_fullsized":"2024-03-22T19:09:13.627015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527730+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:14.527730+0000","last_peered":"2024-03-22T19:09:14.527730+0000","last_clean":"2024-03-22T19:09:14.527730+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:14.527730+0000","last_undegraded":"2024-03-22T19:09:14.527730+0000","last_fullsized":"2024-03-22T19:09:14.527730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:18.145449+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:18.145449+0000","last_peered":"2024-03-22T19:09:18.145449+0000","last_clean":"2024-03-22T19:09:18.145449+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:18.145449+0000","last_undegraded":"2024-03-22T19:09:18.145449+0000","last_fullsized":"2024-03-22T19:09:18.145449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.155143+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:19.155143+0000","last_peered":"2024-03-22T19:09:19.155143+0000","last_clean":"2024-03-22T19:09:19.155143+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:19.155143+0000","last_undegraded":"2024-03-22T19:09:19.155143+0000","last_fullsized":"2024-03-22T19:09:19.155143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527822+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:14.527822+0000","last_peered":"2024-03-22T19:09:14.527822+0000","last_clean":"2024-03-22T19:09:14.527822+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:14.527822+0000","last_undegraded":"2024-03-22T19:09:14.527822+0000","last_fullsized":"2024-03-22T19:09:14.527822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.154825+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:19.154825+0000","last_peered":"2024-03-22T19:09:19.154825+0000","last_clean":"2024-03-22T19:09:19.154825+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:19.154825+0000","last_undegraded":"2024-03-22T19:09:19.154825+0000","last_fullsized":"2024-03-22T19:09:19.154825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530538+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:16.530538+0000","last_peered":"2024-03-22T19:09:16.530538+0000","last_clean":"2024-03-22T19:09:16.530538+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:16.530538+0000","last_undegraded":"2024-03-22T19:09:16.530538+0000","last_fullsized":"2024-03-22T19:09:16.530538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.141605+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:17.141605+0000","last_peered":"2024-03-22T19:09:17.141605+0000","last_clean":"2024-03-22T19:09:17.141605+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:17.141605+0000","last_undegraded":"2024-03-22T19:09:17.141605+0000","last_fullsized":"2024-03-22T19:09:17.141605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957545+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:14.957545+0000","last_peered":"2024-03-22T19:09:14.957545+0000","last_clean":"2024-03-22T19:09:14.957545+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:14.957545+0000","last_undegraded":"2024-03-22T19:09:14.957545+0000","last_fullsized":"2024-03-22T19:09:14.957545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.142313+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:17.142313+0000","last_peered":"2024-03-22T19:09:17.142313+0000","last_clean":"2024-03-22T19:09:17.142313+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:17.142313+0000","last_undegraded":"2024-03-22T19:09:17.142313+0000","last_fullsized":"2024-03-22T19:09:17.142313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.643928+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:14.643928+0000","last_peered":"2024-03-22T19:09:14.643928+0000","last_clean":"2024-03-22T19:09:14.643928+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:14.643928+0000","last_undegraded":"2024-03-22T19:09:14.643928+0000","last_fullsized":"2024-03-22T19:09:14.643928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.166485+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:20.166485+0000","last_peered":"2024-03-22T19:09:20.166485+0000","last_clean":"2024-03-22T19:09:20.166485+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:20.166485+0000","last_undegraded":"2024-03-22T19:09:20.166485+0000","last_fullsized":"2024-03-22T19:09:20.166485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804419+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:15.804419+0000","last_peered":"2024-03-22T19:09:15.804419+0000","last_clean":"2024-03-22T19:09:15.804419+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:15.804419+0000","last_undegraded":"2024-03-22T19:09:15.804419+0000","last_fullsized":"2024-03-22T19:09:15.804419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.153605+0000","last_change":"2024-03-22T19:09:13.144416+0000","last_active":"2024-03-22T19:09:19.153605+0000","last_peered":"2024-03-22T19:09:19.153605+0000","last_clean":"2024-03-22T19:09:19.153605+0000","last_became_active":"2024-03-22T19:09:13.144063+0000","last_became_peered":"2024-03-22T19:09:13.144063+0000","last_unstale":"2024-03-22T19:09:19.153605+0000","last_undegraded":"2024-03-22T19:09:19.153605+0000","last_fullsized":"2024-03-22T19:09:19.153605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:56:53.881418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530451+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:16.530451+0000","last_peered":"2024-03-22T19:09:16.530451+0000","last_clean":"2024-03-22T19:09:16.530451+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:16.530451+0000","last_undegraded":"2024-03-22T19:09:16.530451+0000","last_fullsized":"2024-03-22T19:09:16.530451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:17.143494+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:17.143494+0000","last_peered":"2024-03-22T19:09:17.143494+0000","last_clean":"2024-03-22T19:09:17.143494+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:17.143494+0000","last_undegraded":"2024-03-22T19:09:17.143494+0000","last_fullsized":"2024-03-22T19:09:17.143494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630990+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:14.630990+0000","last_peered":"2024-03-22T19:09:14.630990+0000","last_clean":"2024-03-22T19:09:14.630990+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:14.630990+0000","last_undegraded":"2024-03-22T19:09:14.630990+0000","last_fullsized":"2024-03-22T19:09:14.630990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:19.153279+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:09:19.153279+0000","last_peered":"2024-03-22T19:09:19.153279+0000","last_clean":"2024-03-22T19:09:19.153279+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:19.153279+0000","last_undegraded":"2024-03-22T19:09:19.153279+0000","last_fullsized":"2024-03-22T19:09:19.153279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631182+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:14.631182+0000","last_peered":"2024-03-22T19:09:14.631182+0000","last_clean":"2024-03-22T19:09:14.631182+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:14.631182+0000","last_undegraded":"2024-03-22T19:09:14.631182+0000","last_fullsized":"2024-03-22T19:09:14.631182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:20.165201+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:20.165201+0000","last_peered":"2024-03-22T19:09:20.165201+0000","last_clean":"2024-03-22T19:09:20.165201+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:20.165201+0000","last_undegraded":"2024-03-22T19:09:20.165201+0000","last_fullsized":"2024-03-22T19:09:20.165201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.599826+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:16.599826+0000","last_peered":"2024-03-22T19:09:16.599826+0000","last_clean":"2024-03-22T19:09:16.599826+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:16.599826+0000","last_undegraded":"2024-03-22T19:09:16.599826+0000","last_fullsized":"2024-03-22T19:09:16.599826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.630530+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:14.630530+0000","last_peered":"2024-03-22T19:09:14.630530+0000","last_clean":"2024-03-22T19:09:14.630530+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:14.630530+0000","last_undegraded":"2024-03-22T19:09:14.630530+0000","last_fullsized":"2024-03-22T19:09:14.630530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.631280+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:14.631280+0000","last_peered":"2024-03-22T19:09:14.631280+0000","last_clean":"2024-03-22T19:09:14.631280+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:14.631280+0000","last_undegraded":"2024-03-22T19:09:14.631280+0000","last_fullsized":"2024-03-22T19:09:14.631280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804448+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:15.804448+0000","last_peered":"2024-03-22T19:09:15.804448+0000","last_clean":"2024-03-22T19:09:15.804448+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:15.804448+0000","last_undegraded":"2024-03-22T19:09:15.804448+0000","last_fullsized":"2024-03-22T19:09:15.804448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924566+0000","last_change":"2024-03-22T19:09:13.650295+0000","last_active":"2024-03-22T19:09:14.924566+0000","last_peered":"2024-03-22T19:09:14.924566+0000","last_clean":"2024-03-22T19:09:14.924566+0000","last_became_active":"2024-03-22T19:09:13.649889+0000","last_became_peered":"2024-03-22T19:09:13.649889+0000","last_unstale":"2024-03-22T19:09:14.924566+0000","last_undegraded":"2024-03-22T19:09:14.924566+0000","last_fullsized":"2024-03-22T19:09:14.924566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:25:15.931309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924757+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:14.924757+0000","last_peered":"2024-03-22T19:09:14.924757+0000","last_clean":"2024-03-22T19:09:14.924757+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:14.924757+0000","last_undegraded":"2024-03-22T19:09:14.924757+0000","last_fullsized":"2024-03-22T19:09:14.924757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.957320+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:14.957320+0000","last_peered":"2024-03-22T19:09:14.957320+0000","last_clean":"2024-03-22T19:09:14.957320+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:14.957320+0000","last_undegraded":"2024-03-22T19:09:14.957320+0000","last_fullsized":"2024-03-22T19:09:14.957320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527838+0000","last_change":"2024-03-22T19:09:13.648683+0000","last_active":"2024-03-22T19:09:14.527838+0000","last_peered":"2024-03-22T19:09:14.527838+0000","last_clean":"2024-03-22T19:09:14.527838+0000","last_became_active":"2024-03-22T19:09:13.648300+0000","last_became_peered":"2024-03-22T19:09:13.648300+0000","last_unstale":"2024-03-22T19:09:14.527838+0000","last_undegraded":"2024-03-22T19:09:14.527838+0000","last_fullsized":"2024-03-22T19:09:14.527838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:24:31.803987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804480+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:15.804480+0000","last_peered":"2024-03-22T19:09:15.804480+0000","last_clean":"2024-03-22T19:09:15.804480+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:15.804480+0000","last_undegraded":"2024-03-22T19:09:15.804480+0000","last_fullsized":"2024-03-22T19:09:15.804480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.924950+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:14.924950+0000","last_peered":"2024-03-22T19:09:14.924950+0000","last_clean":"2024-03-22T19:09:14.924950+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:14.924950+0000","last_undegraded":"2024-03-22T19:09:14.924950+0000","last_fullsized":"2024-03-22T19:09:14.924950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530519+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:16.530519+0000","last_peered":"2024-03-22T19:09:16.530519+0000","last_clean":"2024-03-22T19:09:16.530519+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:16.530519+0000","last_undegraded":"2024-03-22T19:09:16.530519+0000","last_fullsized":"2024-03-22T19:09:16.530519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.942745+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:14.942745+0000","last_peered":"2024-03-22T19:09:14.942745+0000","last_clean":"2024-03-22T19:09:14.942745+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:14.942745+0000","last_undegraded":"2024-03-22T19:09:14.942745+0000","last_fullsized":"2024-03-22T19:09:14.942745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:16.530420+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:16.530420+0000","last_peered":"2024-03-22T19:09:16.530420+0000","last_clean":"2024-03-22T19:09:16.530420+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:16.530420+0000","last_undegraded":"2024-03-22T19:09:16.530420+0000","last_fullsized":"2024-03-22T19:09:16.530420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:13.647757+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:13.647757+0000","last_peered":"2024-03-22T19:09:13.647757+0000","last_clean":"2024-03-22T19:09:13.647757+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:13.647757+0000","last_undegraded":"2024-03-22T19:09:13.647757+0000","last_fullsized":"2024-03-22T19:09:13.647757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:14.527795+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:14.527795+0000","last_peered":"2024-03-22T19:09:14.527795+0000","last_clean":"2024-03-22T19:09:14.527795+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:14.527795+0000","last_undegraded":"2024-03-22T19:09:14.527795+0000","last_fullsized":"2024-03-22T19:09:14.527795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T19:09:15.804462+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:15.804462+0000","last_peered":"2024-03-22T19:09:15.804462+0000","last_clean":"2024-03-22T19:09:15.804462+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:15.804462+0000","last_undegraded":"2024-03-22T19:09:15.804462+0000","last_fullsized":"2024-03-22T19:09:15.804462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7368,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":169,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":40048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":413367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776141,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":1004780,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939666,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":413345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.63},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.88800000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.748},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.85399999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168599,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":412630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.439}]},{"osd":1,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.627}]},{"osd":3,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.751}]},{"osd":5,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.5},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.74099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397530,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":1005817,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":412313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72099999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.443}]},{"osd":1,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81699999999999995,"5min":0.875,"15min":0.875},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.369,"15min":1.369},"last":0.746}]},{"osd":2,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.76200000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.53400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71399999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78900000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.749,"15min":0.76100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84599999999999997,"5min":1.55,"15min":1.55},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85799999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Mar 22 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.75900000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.81599999999999995}]},{"osd":9,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.63100000000000001}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":413478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.42599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72599999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.71799999999999997,"15min":0.79400000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":6.71,"15min":6.71},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Mar 22 19:08:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Mar 22 19:08:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.26600000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.375},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:08:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40699999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68100000000000005}]},{"osd":9,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.628}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738419,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":414353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:08:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:08:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45800000000000002,"15min":0.44400000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.372}]},{"osd":4,"last update":"Fri Mar 22 19:08:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:08:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96099999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:08:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.60999999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":421656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Mar 22 19:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.502},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Mar 22 19:08:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.51500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:08:43 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.44700000000000001}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60499999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215144,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":412770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78000000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.42699999999999999},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.024,"5min":1.244,"15min":1.244},"last":0.623},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:08:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1359999999999999,"5min":1.367,"15min":1.367},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0940000000000001,"5min":1.208,"15min":1.208},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Mar 22 19:08:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69899999999999995,"15min":0.70399999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93700000000000006,"5min":1.417,"15min":1.417},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.59699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1080000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Mar 22 19:09:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.077,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94299999999999995,"5min":1.641,"15min":1.641},"last":0.45800000000000002}]},{"osd":7,"last update":"Fri Mar 22 19:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.74399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953509,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":412819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38,"15min":0.377},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:08:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:09:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.628},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.92400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:08:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73699999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.90400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":5,"up_from":37,"seq":158913789986,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1005541,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.71299999999999997,"15min":0.73899999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82399999999999995,"5min":3.887,"15min":3.887},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.49199999999999999}]},{"osd":1,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36599999999999999},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:08:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.76300000000000001,"15min":0.78700000000000003},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83999999999999997,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri Mar 22 19:08:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69299999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.86699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:09:23.505 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T19:09:23.506 INFO:tasks.cephadm:Setting up client nodes... 2024-03-22T19:09:23.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T19:09:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:23 smithi167 ceph-mon[47728]: pgmap v236: 97 pgs: 97 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 960 B/s rd, 6.2 KiB/s wr, 21 op/s 2024-03-22T19:09:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:23 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3725279498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:24.385 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:24 smithi086 ceph-mon[38032]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:09:24.385 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:24 smithi086 ceph-mon[38032]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:09:24.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:24 smithi131 ceph-mon[78559]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:09:24.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:24 smithi131 ceph-mon[78559]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:09:24.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:24 smithi167 ceph-mon[47728]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:09:24.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:24 smithi167 ceph-mon[47728]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 -- 172.21.15.86:0/803102644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 msgr2=0x7fc0a01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 --2- 172.21.15.86:0/803102644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a01011b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc0940099c0 tx=0x7fc094009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 -- 172.21.15.86:0/803102644 shutdown_connections 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 --2- 172.21.15.86:0/803102644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 0x7fc0a01091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 --2- 172.21.15.86:0/803102644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a01011b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 --2- 172.21.15.86:0/803102644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0101f90 0x7fc0a01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 -- 172.21.15.86:0/803102644 >> 172.21.15.86:0/803102644 conn(0x7fc0a00fc330 msgr2=0x7fc0a00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 -- 172.21.15.86:0/803102644 shutdown_connections 2024-03-22T19:09:25.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.134+0000 7fc0a8800700 1 -- 172.21.15.86:0/803102644 wait complete. 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 Processor -- start 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 -- start start 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a0196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0101f90 0x7fc0a0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 0x7fc0a019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019e7c0 con 0x7fc0a0100d90 2024-03-22T19:09:25.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019e900 con 0x7fc0a0101f90 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.135+0000 7fc0a8800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019eac0 con 0x7fc0a0102930 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a0196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a0196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43096/0 (socket says 172.21.15.86:43096) 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 -- 172.21.15.86:0/3214414856 learned_addr learned my addr 172.21.15.86:0/3214414856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5400700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0101f90 0x7fc0a0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a6800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 0x7fc0a019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 -- 172.21.15.86:0/3214414856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 msgr2=0x7fc0a019e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 0x7fc0a019e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 -- 172.21.15.86:0/3214414856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0101f90 msgr2=0x7fc0a0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0101f90 0x7fc0a0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 -- 172.21.15.86:0/3214414856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940095f0 con 0x7fc0a0100d90 2024-03-22T19:09:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a6800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 0x7fc0a019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.136+0000 7fc0a5e00700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a0196d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc094003730 tx=0x7fc09400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.137+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09401d070 con 0x7fc0a0100d90 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.137+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc094005510 con 0x7fc0a0100d90 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.137+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094004a00 con 0x7fc0a0100d90 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.137+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a019ed40 con 0x7fc0a0100d90 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.137+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a019f190 con 0x7fc0a0100d90 2024-03-22T19:09:25.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.138+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a0108260 con 0x7fc0a0100d90 2024-03-22T19:09:25.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.140+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc094004ba0 con 0x7fc0a0100d90 2024-03-22T19:09:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.141+0000 7fc09aa00700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0880a0ba0 0x7fc0880a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.141+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0940ccd60 con 0x7fc0a0100d90 2024-03-22T19:09:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.141+0000 7fc0a5400700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0880a0ba0 0x7fc0880a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.142+0000 7fc0a5400700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0880a0ba0 0x7fc0880a3060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc090009c60 tx=0x7fc090009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.144+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc094096710 con 0x7fc0a0100d90 2024-03-22T19:09:25.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.424+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc0a002ce40 con 0x7fc0a0100d90 2024-03-22T19:09:25.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.429+0000 7fc09aa00700 1 -- 172.21.15.86:0/3214414856 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc094083e60 con 0x7fc0a0100d90 2024-03-22T19:09:25.430 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T19:09:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBl1/1laVJaGRAATay1FPpDPdnXHMDab4uPxQ== 2024-03-22T19:09:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.431+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0880a0ba0 msgr2=0x7fc0880a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.431+0000 7fc0a8800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0880a0ba0 0x7fc0880a3060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc090009c60 tx=0x7fc090009260 comp rx=0 tx=0).stop 2024-03-22T19:09:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.431+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 msgr2=0x7fc0a0196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.431+0000 7fc0a8800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a0196d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc094003730 tx=0x7fc09400f690 comp rx=0 tx=0).stop 2024-03-22T19:09:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 shutdown_connections 2024-03-22T19:09:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0880a0ba0 0x7fc0880a3060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0102930 0x7fc0a019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0a0100d90 0x7fc0a0196d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 --2- 172.21.15.86:0/3214414856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0101f90 0x7fc0a0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 >> 172.21.15.86:0/3214414856 conn(0x7fc0a00fc330 msgr2=0x7fc0a01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 shutdown_connections 2024-03-22T19:09:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:25.432+0000 7fc0a8800700 1 -- 172.21.15.86:0/3214414856 wait complete. 2024-03-22T19:09:25.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:25 smithi086 ceph-mon[38032]: pgmap v237: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 21 op/s 2024-03-22T19:09:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:25 smithi131 ceph-mon[78559]: pgmap v237: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 21 op/s 2024-03-22T19:09:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:25 smithi167 ceph-mon[47728]: pgmap v237: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 21 op/s 2024-03-22T19:09:25.849 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:09:25.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-22T19:09:25.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-22T19:09:25.944 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-22T19:09:25.945 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-22T19:09:25.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-22T19:09:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:26.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:26 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3214414856' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T19:09:26.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:26 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3214414856' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T19:09:26.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:26 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3214414856' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T19:09:26.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:26 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3214414856' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T19:09:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:26 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3214414856' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T19:09:26.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:26 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3214414856' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T19:09:27.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:27 smithi086 ceph-mon[38032]: pgmap v238: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-03-22T19:09:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:27 smithi131 ceph-mon[78559]: pgmap v238: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-03-22T19:09:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:27 smithi167 ceph-mon[47728]: pgmap v238: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-03-22T19:09:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.641+0000 7f4e88c00700 1 -- 172.21.15.86:0/1607671426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84100fb0 msgr2=0x7f4e84101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.641+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1607671426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84100fb0 0x7f4e84101390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009990 tx=0x7f4e74009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 -- 172.21.15.86:0/1607671426 shutdown_connections 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1607671426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e84106090 0x7f4e8410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1607671426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84100fb0 0x7f4e84101390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1607671426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e84101960 0x7f4e841059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 -- 172.21.15.86:0/1607671426 >> 172.21.15.86:0/1607671426 conn(0x7f4e840fc800 msgr2=0x7f4e840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 -- 172.21.15.86:0/1607671426 shutdown_connections 2024-03-22T19:09:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.642+0000 7f4e88c00700 1 -- 172.21.15.86:0/1607671426 wait complete. 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 Processor -- start 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 -- start start 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e84100fb0 0x7f4e840730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e84101960 0x7f4e840735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 0x7f4e84073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84075020 con 0x7f4e84106090 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e841a9bf0 con 0x7f4e84100fb0 2024-03-22T19:09:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.643+0000 7f4e88c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e841a9d60 con 0x7f4e84101960 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 0x7f4e84073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 0x7f4e84073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43104/0 (socket says 172.21.15.86:43104) 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 -- 172.21.15.86:0/1105567490 learned_addr learned my addr 172.21.15.86:0/1105567490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e81e00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e84100fb0 0x7f4e840730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e81400700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e84101960 0x7f4e840735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 -- 172.21.15.86:0/1105567490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e84101960 msgr2=0x7f4e840735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e84101960 0x7f4e840735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 -- 172.21.15.86:0/1105567490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e84100fb0 msgr2=0x7f4e840730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e84100fb0 0x7f4e840730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 -- 172.21.15.86:0/1105567490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740095f0 con 0x7f4e84106090 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e82800700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 0x7f4e84073b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c007ee0 tx=0x7f4e7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.644+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c017070 con 0x7f4e84106090 2024-03-22T19:09:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.645+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e7c00f040 con 0x7f4e84106090 2024-03-22T19:09:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.645+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e841aa040 con 0x7f4e84106090 2024-03-22T19:09:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.645+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0054b0 con 0x7f4e84106090 2024-03-22T19:09:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.645+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e841aa510 con 0x7f4e84106090 2024-03-22T19:09:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.646+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e84109350 con 0x7f4e84106090 2024-03-22T19:09:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.648+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f4e7c004ad0 con 0x7f4e84106090 2024-03-22T19:09:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.649+0000 7f4e7aa00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4e680a0b20 0x7f4e680a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.649+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4e7c0cb800 con 0x7f4e84106090 2024-03-22T19:09:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.649+0000 7f4e81e00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4e680a0b20 0x7f4e680a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.650+0000 7f4e81e00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4e680a0b20 0x7f4e680a2fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006010 tx=0x7f4e7400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.653+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4e7c095120 con 0x7f4e84106090 2024-03-22T19:09:27.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.913+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4e84074540 con 0x7f4e84106090 2024-03-22T19:09:27.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.918+0000 7f4e7aa00700 1 -- 172.21.15.86:0/1105567490 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259468 (secure 0 0 0) 0x7f4e7c098740 con 0x7f4e84106090 2024-03-22T19:09:27.922 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4e680a0b20 msgr2=0x7f4e680a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4e680a0b20 0x7f4e680a2fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006010 tx=0x7f4e7400b260 comp rx=0 tx=0).stop 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 msgr2=0x7f4e84073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 0x7f4e84073b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c007ee0 tx=0x7f4e7c00d3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 shutdown_connections 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4e680a0b20 0x7f4e680a2fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84106090 0x7f4e84073b30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e84100fb0 0x7f4e840730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 --2- 172.21.15.86:0/1105567490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e84101960 0x7f4e840735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 >> 172.21.15.86:0/1105567490 conn(0x7f4e840fc800 msgr2=0x7f4e840fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 shutdown_connections 2024-03-22T19:09:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:27.922+0000 7f4e88c00700 1 -- 172.21.15.86:0/1105567490 wait complete. 2024-03-22T19:09:28.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:28 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1105567490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T19:09:28.304 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1126645338},{"type":"v1","addr":"172.21.15.86:6801","nonce":1126645338}]},"active_addr":"172.21.15.86:6801/1126645338","active_change":"2024-03-22T19:03:50.084202+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1673116221}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":375941103}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2025404564}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":4038820216}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3874065762}]}]} 2024-03-22T19:09:28.309 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-22T19:09:28.309 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-22T19:09:28.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T19:09:28.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:28 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1105567490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T19:09:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:28 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1105567490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T19:09:28.666 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:29.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:29 smithi131 ceph-mon[78559]: pgmap v239: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-03-22T19:09:29.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:29 smithi086 ceph-mon[38032]: pgmap v239: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-03-22T19:09:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:29 smithi167 ceph-mon[47728]: pgmap v239: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-03-22T19:09:29.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 -- 172.21.15.86:0/1265316414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c106090 msgr2=0x7f4c9c10dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:29.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/1265316414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c106090 0x7f4c9c10dc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400b0a0 tx=0x7f4c9400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:29.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 -- 172.21.15.86:0/1265316414 shutdown_connections 2024-03-22T19:09:29.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/1265316414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c106090 0x7f4c9c10dc60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:29.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/1265316414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:29.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/1265316414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101960 0x7f4c9c1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:29.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 -- 172.21.15.86:0/1265316414 >> 172.21.15.86:0/1265316414 conn(0x7f4c9c0fc800 msgr2=0x7f4c9c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:29.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 -- 172.21.15.86:0/1265316414 shutdown_connections 2024-03-22T19:09:29.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.907+0000 7f4ca2a00700 1 -- 172.21.15.86:0/1265316414 wait complete. 2024-03-22T19:09:29.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.908+0000 7f4ca2a00700 1 Processor -- start 2024-03-22T19:09:29.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.908+0000 7f4ca2a00700 1 -- start start 2024-03-22T19:09:29.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:29.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca2a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c101960 0x7f4c9c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c106090 0x7f4c9c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca2a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c19e3d0 con 0x7f4c9c106090 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca2a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c19e540 con 0x7f4c9c100fb0 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca2a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c19e700 con 0x7f4c9c101960 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4c9be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4ca0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c106090 0x7f4c9c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4c9be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c1994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50500/0 (socket says 172.21.15.86:50500) 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4c9be00700 1 -- 172.21.15.86:0/2734859351 learned_addr learned my addr 172.21.15.86:0/2734859351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4c93400700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c101960 0x7f4c9c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.909+0000 7f4c9be00700 1 -- 172.21.15.86:0/2734859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c101960 msgr2=0x7f4c9c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c9be00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c101960 0x7f4c9c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c9be00700 1 -- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c106090 msgr2=0x7f4c9c19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:29.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c9be00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c106090 0x7f4c9c19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c9be00700 1 -- 172.21.15.86:0/2734859351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009520 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c93400700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c101960 0x7f4c9c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c9be00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c1994d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00eb30 tx=0x7f4c8c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c009880 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c84009520 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c8c0044e0 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003d80 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.910+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c94009d00 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.911+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c102490 con 0x7f4c9c100fb0 2024-03-22T19:09:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.913+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f4c8c003f20 con 0x7f4c9c100fb0 2024-03-22T19:09:29.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.914+0000 7f4c99600700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c800a0b20 0x7f4c800a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:29.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.914+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4c8c014070 con 0x7f4c9c100fb0 2024-03-22T19:09:29.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.915+0000 7f4c93400700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c800a0b20 0x7f4c800a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:29.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.915+0000 7f4c93400700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c800a0b20 0x7f4c800a2fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c19aa30 tx=0x7f4c84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:29.917+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4c8c096370 con 0x7f4c9c100fb0 2024-03-22T19:09:30.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.122+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c9c04ea80 con 0x7f4c9c100fb0 2024-03-22T19:09:30.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.124+0000 7f4c99600700 1 -- 172.21.15.86:0/2734859351 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f4c8c099990 con 0x7f4c9c100fb0 2024-03-22T19:09:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:30.125 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:14.621457+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:09:30.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c800a0b20 msgr2=0x7f4c800a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:30.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c800a0b20 0x7f4c800a2fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c19aa30 tx=0x7f4c84009260 comp rx=0 tx=0).stop 2024-03-22T19:09:30.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 msgr2=0x7f4c9c1994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:30.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c1994d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00eb30 tx=0x7f4c8c00ee40 comp rx=0 tx=0).stop 2024-03-22T19:09:30.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 shutdown_connections 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c800a0b20 0x7f4c800a2fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c106090 0x7f4c9c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100fb0 0x7f4c9c1994d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 --2- 172.21.15.86:0/2734859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c101960 0x7f4c9c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 >> 172.21.15.86:0/2734859351 conn(0x7f4c9c0fc800 msgr2=0x7f4c9c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 shutdown_connections 2024-03-22T19:09:30.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:30.127+0000 7f4ca2a00700 1 -- 172.21.15.86:0/2734859351 wait complete. 2024-03-22T19:09:30.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:30 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2734859351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:30.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:30 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2734859351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:30.549 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-22T19:09:30.549 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T19:09:30.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:30 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2734859351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:30.904 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:31.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:31 smithi086 ceph-mon[38032]: pgmap v240: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-03-22T19:09:31.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:31 smithi131 ceph-mon[78559]: pgmap v240: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-03-22T19:09:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:31 smithi167 ceph-mon[47728]: pgmap v240: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-03-22T19:09:32.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 -- 172.21.15.86:0/1003302467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040106090 msgr2=0x7f704010dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:32.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 --2- 172.21.15.86:0/1003302467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040106090 0x7f704010dc60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f703800b0a0 tx=0x7f703800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 -- 172.21.15.86:0/1003302467 shutdown_connections 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 --2- 172.21.15.86:0/1003302467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040106090 0x7f704010dc60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 --2- 172.21.15.86:0/1003302467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040100fb0 0x7f7040101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 --2- 172.21.15.86:0/1003302467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040101960 0x7f70401059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 -- 172.21.15.86:0/1003302467 >> 172.21.15.86:0/1003302467 conn(0x7f70400fc800 msgr2=0x7f70400fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 -- 172.21.15.86:0/1003302467 shutdown_connections 2024-03-22T19:09:32.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.114+0000 7f7044e00700 1 -- 172.21.15.86:0/1003302467 wait complete. 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.115+0000 7f7044e00700 1 Processor -- start 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 -- start start 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 0x7f70400730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 0x7f70400735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040106090 0x7f7040073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70401a56f0 con 0x7f7040101960 2024-03-22T19:09:32.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70401a5860 con 0x7f7040100fb0 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f7044e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70401a5a20 con 0x7f7040106090 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 0x7f70400735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 0x7f70400730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040106090 0x7f7040073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 0x7f70400730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50512/0 (socket says 172.21.15.86:50512) 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 0x7f70400735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55664/0 (socket says 172.21.15.86:55664) 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040106090 0x7f7040073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47564/0 (socket says 172.21.15.86:47564) 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.116+0000 7f703de00700 1 -- 172.21.15.86:0/2273283424 learned_addr learned my addr 172.21.15.86:0/2273283424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703d400700 1 -- 172.21.15.86:0/2273283424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040106090 msgr2=0x7f7040073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703d400700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040106090 0x7f7040073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703d400700 1 -- 172.21.15.86:0/2273283424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 msgr2=0x7f70400730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703d400700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 0x7f70400730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703d400700 1 -- 172.21.15.86:0/2273283424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038009d00 con 0x7f7040101960 2024-03-22T19:09:32.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703de00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 0x7f70400730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:32.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f703d400700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 0x7f70400735f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f702c00d720 tx=0x7f702c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:32.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c004020 con 0x7f7040101960 2024-03-22T19:09:32.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702c004830 con 0x7f7040101960 2024-03-22T19:09:32.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70401a5d00 con 0x7f7040101960 2024-03-22T19:09:32.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0054f0 con 0x7f7040101960 2024-03-22T19:09:32.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.117+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70401a61b0 con 0x7f7040101960 2024-03-22T19:09:32.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.118+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70401093b0 con 0x7f7040101960 2024-03-22T19:09:32.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.120+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f702c0049a0 con 0x7f7040101960 2024-03-22T19:09:32.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.122+0000 7f7036a00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70240a0b20 0x7f70240a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:32.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.122+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f702c0cbc80 con 0x7f7040101960 2024-03-22T19:09:32.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.122+0000 7f703de00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70240a0b20 0x7f70240a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:32.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.122+0000 7f703de00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70240a0b20 0x7f70240a2fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7030011590 tx=0x7f7030009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:32.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.125+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f702c0955a0 con 0x7f7040101960 2024-03-22T19:09:32.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.330+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7040074540 con 0x7f7040101960 2024-03-22T19:09:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.332+0000 7f7036a00700 1 -- 172.21.15.86:0/2273283424 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f702c01d090 con 0x7f7040101960 2024-03-22T19:09:32.333 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:09:32.333 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:14.621457+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:09:32.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.334+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70240a0b20 msgr2=0x7f70240a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.334+0000 7f7044e00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70240a0b20 0x7f70240a2fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7030011590 tx=0x7f7030009260 comp rx=0 tx=0).stop 2024-03-22T19:09:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.334+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 msgr2=0x7f70400735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.334+0000 7f7044e00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 0x7f70400735f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f702c00d720 tx=0x7f702c00da30 comp rx=0 tx=0).stop 2024-03-22T19:09:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 shutdown_connections 2024-03-22T19:09:32.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70240a0b20 0x7f70240a2fe0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7040106090 0x7f7040073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7040100fb0 0x7f70400730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 --2- 172.21.15.86:0/2273283424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040101960 0x7f70400735f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:32.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 >> 172.21.15.86:0/2273283424 conn(0x7f70400fc800 msgr2=0x7f70400feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 shutdown_connections 2024-03-22T19:09:32.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:32.335+0000 7f7044e00700 1 -- 172.21.15.86:0/2273283424 wait complete. 2024-03-22T19:09:32.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-22T19:09:32.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-22T19:09:32.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-22T19:09:32.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-22T19:09:32.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-22T19:09:32.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-22T19:09:32.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-22T19:09:32.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-22T19:09:32.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-22T19:09:32.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-22T19:09:32.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-22T19:09:32.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-22T19:09:33.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:33 smithi131 ceph-mon[78559]: pgmap v241: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-03-22T19:09:33.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2273283424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:33.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:33 smithi086 ceph-mon[38032]: pgmap v241: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-03-22T19:09:33.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2273283424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:33 smithi167 ceph-mon[47728]: pgmap v241: 97 pgs: 97 active+clean; 626 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-03-22T19:09:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2273283424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:09:34.563 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:34.615 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:34.716 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.328 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:35 smithi086 ceph-mon[38032]: pgmap v242: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-03-22T19:09:35.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:35 smithi131 ceph-mon[78559]: pgmap v242: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-03-22T19:09:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:35 smithi167 ceph-mon[47728]: pgmap v242: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-03-22T19:09:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.593 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:37.345 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:37 smithi086 ceph-mon[38032]: pgmap v243: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-03-22T19:09:37.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:37 smithi131 ceph-mon[78559]: pgmap v243: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-03-22T19:09:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:37 smithi167 ceph-mon[47728]: pgmap v243: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-03-22T19:09:37.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 -- 172.21.15.86:0/3537715107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 msgr2=0x7f31601011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 --2- 172.21.15.86:0/3537715107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31601011c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3150009990 tx=0x7f3150009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 -- 172.21.15.86:0/3537715107 shutdown_connections 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 --2- 172.21.15.86:0/3537715107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3160105ec0 0x7f316010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 --2- 172.21.15.86:0/3537715107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31601011c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 --2- 172.21.15.86:0/3537715107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3160101790 0x7f31601057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 -- 172.21.15.86:0/3537715107 >> 172.21.15.86:0/3537715107 conn(0x7f31600fc6b0 msgr2=0x7f31600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 -- 172.21.15.86:0/3537715107 shutdown_connections 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.607+0000 7f315f200700 1 -- 172.21.15.86:0/3537715107 wait complete. 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.608+0000 7f315f200700 1 Processor -- start 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.608+0000 7f315f200700 1 -- start start 2024-03-22T19:09:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31600730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315f200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3160101790 0x7f31600735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3160105ec0 0x7f3160073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601a53f0 con 0x7f3160100de0 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601a5560 con 0x7f3160105ec0 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315f200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601a5720 con 0x7f3160101790 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31600730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31600730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55672/0 (socket says 172.21.15.86:55672) 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 -- 172.21.15.86:0/2130637678 learned_addr learned my addr 172.21.15.86:0/2130637678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315e800700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3160105ec0 0x7f3160073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f3157e00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3160101790 0x7f31600735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 -- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3160101790 msgr2=0x7f31600735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3160101790 0x7f31600735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 -- 172.21.15.86:0/2130637678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3160105ec0 msgr2=0x7f3160073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3160105ec0 0x7f3160073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.609+0000 7f315de00700 1 -- 172.21.15.86:0/2130637678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500095f0 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.610+0000 7f315de00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31600730b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3150005770 tx=0x7f31500057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.610+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315001d070 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.610+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3150021410 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.610+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150017650 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.610+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31601a59a0 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.610+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31601a5df0 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.611+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31601022c0 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.613+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f31500217f0 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.614+0000 7f3156a00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31440a0ba0 0x7f31440a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.614+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31500cd2b0 con 0x7f3160100de0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.614+0000 7f3157e00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31440a0ba0 0x7f31440a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.614+0000 7f3156a00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] conn(0x7f31440aa380 0x7f31440ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:37.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.614+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 --> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31440ace50 con 0x7f31440aa380 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.614+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31500cf890 con 0x7f3160100de0 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.615+0000 7f3157e00700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31440a0ba0 0x7f31440a3060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3160074790 tx=0x7f3148009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.615+0000 7f315e800700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] conn(0x7f31440aa380 0x7f31440ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.615+0000 7f315e800700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] conn(0x7f31440aa380 0x7f31440ac7a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.616+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== osd.11 v2:172.21.15.167:6824/736141441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f31440ace50 con 0x7f31440aa380 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.633+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 --> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3160110d20 con 0x7f31440aa380 2024-03-22T19:09:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.634+0000 7f3156a00700 1 -- 172.21.15.86:0/2130637678 <== osd.11 v2:172.21.15.167:6824/736141441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3160110d20 con 0x7f31440aa380 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] conn(0x7f31440aa380 msgr2=0x7f31440ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] conn(0x7f31440aa380 0x7f31440ac7a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31440a0ba0 msgr2=0x7f31440a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f31440a0ba0 0x7f31440a3060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3160074790 tx=0x7f3148009260 comp rx=0 tx=0).stop 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 msgr2=0x7f31600730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 --2- 172.21.15.86:0/2130637678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3160100de0 0x7f31600730b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3150005770 tx=0x7f31500057a0 comp rx=0 tx=0).stop 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315e800700 1 -- 172.21.15.86:0/2130637678 reap_dead start 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 shutdown_connections 2024-03-22T19:09:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 >> 172.21.15.86:0/2130637678 conn(0x7f31600fc6b0 msgr2=0x7f31600fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 shutdown_connections 2024-03-22T19:09:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:37.635+0000 7f315f200700 1 -- 172.21.15.86:0/2130637678 wait complete. 2024-03-22T19:09:39.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:39 smithi131 ceph-mon[78559]: pgmap v244: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:09:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:39 smithi086 ceph-mon[38032]: pgmap v244: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.574+0000 7fe0f4200700 1 -- 172.21.15.86:0/2819334198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 msgr2=0x7fe0ec068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.574+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2819334198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec068c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009990 tx=0x7fe0d8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 -- 172.21.15.86:0/2819334198 shutdown_connections 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2819334198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ec069140 0x7fe0ec10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2819334198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0ec106ec0 0x7fe0ec1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2819334198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec068c00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 -- 172.21.15.86:0/2819334198 >> 172.21.15.86:0/2819334198 conn(0x7fe0ec076db0 msgr2=0x7fe0ec0771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 -- 172.21.15.86:0/2819334198 shutdown_connections 2024-03-22T19:09:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.575+0000 7fe0f4200700 1 -- 172.21.15.86:0/2819334198 wait complete. 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 Processor -- start 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 -- start start 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ec069140 0x7fe0ec1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0ec106ec0 0x7fe0ec19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19c070 con 0x7fe0ec068780 2024-03-22T19:09:39.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19c1e0 con 0x7fe0ec069140 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f4200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19c3a0 con 0x7fe0ec106ec0 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42784/0 (socket says 172.21.15.86:42784) 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.576+0000 7fe0f1800700 1 -- 172.21.15.86:0/2750374745 learned_addr learned my addr 172.21.15.86:0/2750374745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f0e00700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ec069140 0x7fe0ec1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f2200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0ec106ec0 0x7fe0ec19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f1800700 1 -- 172.21.15.86:0/2750374745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0ec106ec0 msgr2=0x7fe0ec19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f1800700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0ec106ec0 0x7fe0ec19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f1800700 1 -- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ec069140 msgr2=0x7fe0ec1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f1800700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ec069140 0x7fe0ec1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f1800700 1 -- 172.21.15.86:0/2750374745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d80095f0 con 0x7fe0ec068780 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f0e00700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ec069140 0x7fe0ec1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f2200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0ec106ec0 0x7fe0ec19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f1800700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec197170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e9a0 tx=0x7fe0e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0009790 con 0x7fe0ec068780 2024-03-22T19:09:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec19c680 con 0x7fe0ec068780 2024-03-22T19:09:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0e000cbd0 con 0x7fe0ec068780 2024-03-22T19:09:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.577+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0018680 con 0x7fe0ec068780 2024-03-22T19:09:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.578+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec1a3ee0 con 0x7fe0ec068780 2024-03-22T19:09:39.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:39 smithi167 ceph-mon[47728]: pgmap v244: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:09:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.578+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0ec102470 con 0x7fe0ec068780 2024-03-22T19:09:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.580+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fe0e000cd40 con 0x7fe0ec068780 2024-03-22T19:09:39.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0ea000700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe0d40a0ba0 0x7fe0d40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:39.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe0e0014070 con 0x7fe0ec068780 2024-03-22T19:09:39.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0f0e00700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe0d40a0ba0 0x7fe0d40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:39.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0ea000700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] conn(0x7fe0d40aa380 0x7fe0d40ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 --> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0d40ace50 con 0x7fe0d40aa380 2024-03-22T19:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e00ccd40 con 0x7fe0ec068780 2024-03-22T19:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0f0e00700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe0d40a0ba0 0x7fe0d40a3060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800b2e0 tx=0x7fe0d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.582+0000 7fe0f2200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] conn(0x7fe0d40aa380 0x7fe0d40ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.583+0000 7fe0f2200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] conn(0x7fe0d40aa380 0x7fe0d40ac7a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:39.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.584+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== osd.4 v2:172.21.15.131:6808/2968759131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe0d40ace50 con 0x7fe0d40aa380 2024-03-22T19:09:39.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.600+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 --> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0ec110f20 con 0x7fe0d40aa380 2024-03-22T19:09:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0ea000700 1 -- 172.21.15.86:0/2750374745 <== osd.4 v2:172.21.15.131:6808/2968759131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe0ec110f20 con 0x7fe0d40aa380 2024-03-22T19:09:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] conn(0x7fe0d40aa380 msgr2=0x7fe0d40ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] conn(0x7fe0d40aa380 0x7fe0d40ac7a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe0d40a0ba0 msgr2=0x7fe0d40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe0d40a0ba0 0x7fe0d40a3060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800b2e0 tx=0x7fe0d8005fb0 comp rx=0 tx=0).stop 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 msgr2=0x7fe0ec197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 --2- 172.21.15.86:0/2750374745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec068780 0x7fe0ec197170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e9a0 tx=0x7fe0e000ed60 comp rx=0 tx=0).stop 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f2200700 1 -- 172.21.15.86:0/2750374745 reap_dead start 2024-03-22T19:09:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 shutdown_connections 2024-03-22T19:09:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 >> 172.21.15.86:0/2750374745 conn(0x7fe0ec076db0 msgr2=0x7fe0ec0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 shutdown_connections 2024-03-22T19:09:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:39.601+0000 7fe0f4200700 1 -- 172.21.15.86:0/2750374745 wait complete. 2024-03-22T19:09:40.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 -- 172.21.15.86:0/3918376517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 msgr2=0x7f24841059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:40.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 --2- 172.21.15.86:0/3918376517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 0x7f24841059b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2470009990 tx=0x7f2470009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 -- 172.21.15.86:0/3918376517 shutdown_connections 2024-03-22T19:09:40.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 --2- 172.21.15.86:0/3918376517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484106090 0x7f248410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 --2- 172.21.15.86:0/3918376517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2484100fb0 0x7f2484101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 --2- 172.21.15.86:0/3918376517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 0x7f24841059b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 -- 172.21.15.86:0/3918376517 >> 172.21.15.86:0/3918376517 conn(0x7f24840fc800 msgr2=0x7f24840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:40.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 -- 172.21.15.86:0/3918376517 shutdown_connections 2024-03-22T19:09:40.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.243+0000 7f2489600700 1 -- 172.21.15.86:0/3918376517 wait complete. 2024-03-22T19:09:40.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.244+0000 7f2489600700 1 Processor -- start 2024-03-22T19:09:40.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.244+0000 7f2489600700 1 -- start start 2024-03-22T19:09:40.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2489600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484100fb0 0x7f2484199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:40.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2489600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 0x7f2484199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2489600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2484106090 0x7f248419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2489600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e300 con 0x7f2484101960 2024-03-22T19:09:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2489600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e470 con 0x7f2484106090 2024-03-22T19:09:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2489600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e630 con 0x7f2484100fb0 2024-03-22T19:09:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 0x7f2484199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484100fb0 0x7f2484199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484100fb0 0x7f2484199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:56334/0 (socket says 172.21.15.86:56334) 2024-03-22T19:09:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482a00700 1 -- 172.21.15.86:0/1115729301 learned_addr learned my addr 172.21.15.86:0/1115729301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2483400700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2484106090 0x7f248419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482000700 1 -- 172.21.15.86:0/1115729301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484100fb0 msgr2=0x7f2484199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482000700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484100fb0 0x7f2484199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482000700 1 -- 172.21.15.86:0/1115729301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2484106090 msgr2=0x7f248419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482000700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2484106090 0x7f248419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482000700 1 -- 172.21.15.86:0/1115729301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24700095f0 con 0x7f2484101960 2024-03-22T19:09:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.245+0000 7f2482a00700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2484100fb0 0x7f2484199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.246+0000 7f2482000700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 0x7f2484199940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2470000c00 tx=0x7f247000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.246+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247001d070 con 0x7f2484101960 2024-03-22T19:09:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.246+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2470005030 con 0x7f2484101960 2024-03-22T19:09:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.246+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248419e8b0 con 0x7f2484101960 2024-03-22T19:09:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.246+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700049b0 con 0x7f2484101960 2024-03-22T19:09:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.246+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248419eb50 con 0x7f2484101960 2024-03-22T19:09:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.247+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2484110f90 con 0x7f2484101960 2024-03-22T19:09:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.249+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f2470005410 con 0x7f2484101960 2024-03-22T19:09:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.250+0000 7f247b400700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24680a0b20 0x7f24680a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.250+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24700cd550 con 0x7f2484101960 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.250+0000 7f2482a00700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24680a0b20 0x7f24680a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.251+0000 7f247b400700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] conn(0x7f24680aa300 0x7f24680ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.251+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 --> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24680acdd0 con 0x7f24680aa300 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.251+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24700cd8c0 con 0x7f2484101960 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.251+0000 7f2483400700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] conn(0x7f24680aa300 0x7f24680ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.251+0000 7f2482a00700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24680a0b20 0x7f24680a2fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24740104e0 tx=0x7f2474007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:40.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.251+0000 7f2483400700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] conn(0x7f24680aa300 0x7f24680ac720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:40.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.252+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== osd.6 v2:172.21.15.86:6818/4292626130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f24680acdd0 con 0x7f24680aa300 2024-03-22T19:09:40.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.268+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 --> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2484074230 con 0x7f24680aa300 2024-03-22T19:09:40.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f247b400700 1 -- 172.21.15.86:0/1115729301 <== osd.6 v2:172.21.15.86:6818/4292626130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2484074230 con 0x7f24680aa300 2024-03-22T19:09:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] conn(0x7f24680aa300 msgr2=0x7f24680ac720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2489600700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] conn(0x7f24680aa300 0x7f24680ac720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24680a0b20 msgr2=0x7f24680a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2489600700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f24680a0b20 0x7f24680a2fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24740104e0 tx=0x7f2474007480 comp rx=0 tx=0).stop 2024-03-22T19:09:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 msgr2=0x7f2484199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2489600700 1 --2- 172.21.15.86:0/1115729301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2484101960 0x7f2484199940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2470000c00 tx=0x7f247000f690 comp rx=0 tx=0).stop 2024-03-22T19:09:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.269+0000 7f2483400700 1 -- 172.21.15.86:0/1115729301 reap_dead start 2024-03-22T19:09:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.270+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 shutdown_connections 2024-03-22T19:09:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.270+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 >> 172.21.15.86:0/1115729301 conn(0x7f24840fc800 msgr2=0x7f24840feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.270+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 shutdown_connections 2024-03-22T19:09:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:40.270+0000 7f2489600700 1 -- 172.21.15.86:0/1115729301 wait complete. 2024-03-22T19:09:41.168 INFO:teuthology.orchestra.run.smithi086.stdout:137438953513 2024-03-22T19:09:41.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-22T19:09:41.298 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:41 smithi086 ceph-mon[38032]: pgmap v245: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-22T19:09:41.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:41 smithi131 ceph-mon[78559]: pgmap v245: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-22T19:09:41.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:41 smithi167 ceph-mon[47728]: pgmap v245: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-22T19:09:42.015 INFO:teuthology.orchestra.run.smithi086.stdout:188978561059 2024-03-22T19:09:42.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 -- 172.21.15.86:0/569881752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d810d5f0 msgr2=0x7f70d810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 --2- 172.21.15.86:0/569881752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d810d5f0 0x7f70d810fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b0a0 tx=0x7f70d000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 -- 172.21.15.86:0/569881752 shutdown_connections 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 --2- 172.21.15.86:0/569881752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d810d5f0 0x7f70d810fa00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 --2- 172.21.15.86:0/569881752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70d8073180 0x7f70d8073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 --2- 172.21.15.86:0/569881752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d8073aa0 0x7f70d810d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 -- 172.21.15.86:0/569881752 >> 172.21.15.86:0/569881752 conn(0x7f70d80fc800 msgr2=0x7f70d80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 -- 172.21.15.86:0/569881752 shutdown_connections 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.179+0000 7f70e0200700 1 -- 172.21.15.86:0/569881752 wait complete. 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.180+0000 7f70e0200700 1 Processor -- start 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 -- start start 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d8073180 0x7f70d810cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d8073aa0 0x7f70d8109ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70d810d5f0 0x7f70d810a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d810ad20 con 0x7f70d810d5f0 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d810ae90 con 0x7f70d8073aa0 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.181+0000 7f70e0200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d806a820 con 0x7f70d8073180 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.182+0000 7f70dd800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d8073180 0x7f70d810cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.182+0000 7f70dd800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d8073180 0x7f70d810cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:56350/0 (socket says 172.21.15.86:56350) 2024-03-22T19:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.182+0000 7f70dd800700 1 -- 172.21.15.86:0/546653797 learned_addr learned my addr 172.21.15.86:0/546653797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.182+0000 7f70de200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70d810d5f0 0x7f70d810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70dce00700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d8073aa0 0x7f70d8109ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70de200700 1 -- 172.21.15.86:0/546653797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d8073180 msgr2=0x7f70d810cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70de200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d8073180 0x7f70d810cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70de200700 1 -- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d8073aa0 msgr2=0x7f70d8109ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70de200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70d8073aa0 0x7f70d8109ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70de200700 1 -- 172.21.15.86:0/546653797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0009d00 con 0x7f70d810d5f0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.183+0000 7f70de200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70d810d5f0 0x7f70d810a550 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b070 tx=0x7f70d0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.184+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d000e040 con 0x7f70d810d5f0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.184+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d0004c50 con 0x7f70d810d5f0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.184+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0005040 con 0x7f70d810d5f0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.184+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d806aaa0 con 0x7f70d810d5f0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.184+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d806af50 con 0x7f70d810d5f0 2024-03-22T19:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.185+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70d81131b0 con 0x7f70d810d5f0 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.188+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f70d001d470 con 0x7f70d810d5f0 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.190+0000 7f70d6000700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70c00a0ba0 0x7f70c00a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.190+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70d00cdc30 con 0x7f70d810d5f0 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.190+0000 7f70d6000700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] conn(0x7f70c00aa380 0x7f70c00ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.190+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 --> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70c00ace50 con 0x7f70c00aa380 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.190+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d00cdfa0 con 0x7f70d810d5f0 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.190+0000 7f70dd800700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70c00a0ba0 0x7f70c00a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.191+0000 7f70dce00700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] conn(0x7f70c00aa380 0x7f70c00ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.191+0000 7f70dd800700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70c00a0ba0 0x7f70c00a3060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005950 tx=0x7f70cc00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.192+0000 7f70dce00700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] conn(0x7f70c00aa380 0x7f70c00ac7a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.192+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== osd.7 v2:172.21.15.131:6816/1354625661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f70c00ace50 con 0x7f70c00aa380 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.212+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 --> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70d8074830 con 0x7f70c00aa380 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.212+0000 7f70d6000700 1 -- 172.21.15.86:0/546653797 <== osd.7 v2:172.21.15.131:6816/1354625661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f70d8074830 con 0x7f70c00aa380 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] conn(0x7f70c00aa380 msgr2=0x7f70c00ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70e0200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] conn(0x7f70c00aa380 0x7f70c00ac7a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70c00a0ba0 msgr2=0x7f70c00a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70e0200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70c00a0ba0 0x7f70c00a3060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005950 tx=0x7f70cc00b500 comp rx=0 tx=0).stop 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70d810d5f0 msgr2=0x7f70d810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70e0200700 1 --2- 172.21.15.86:0/546653797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70d810d5f0 0x7f70d810a550 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b070 tx=0x7f70d0007930 comp rx=0 tx=0).stop 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.213+0000 7f70de200700 1 -- 172.21.15.86:0/546653797 reap_dead start 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.214+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 shutdown_connections 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.214+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 >> 172.21.15.86:0/546653797 conn(0x7f70d80fc800 msgr2=0x7f70d80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.214+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 shutdown_connections 2024-03-22T19:09:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.214+0000 7f70e0200700 1 -- 172.21.15.86:0/546653797 wait complete. 2024-03-22T19:09:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 -- 172.21.15.86:0/1552004210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4106e30 msgr2=0x7f3ce4107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 --2- 172.21.15.86:0/1552004210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4106e30 0x7f3ce4107210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40099c0 tx=0x7f3cd4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 -- 172.21.15.86:0/1552004210 shutdown_connections 2024-03-22T19:09:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 --2- 172.21.15.86:0/1552004210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ce4101820 0x7f3ce410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 --2- 172.21.15.86:0/1552004210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4106e30 0x7f3ce4107210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 --2- 172.21.15.86:0/1552004210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce4100e60 0x7f3ce41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 -- 172.21.15.86:0/1552004210 >> 172.21.15.86:0/1552004210 conn(0x7f3ce4077ed0 msgr2=0x7f3ce40782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 -- 172.21.15.86:0/1552004210 shutdown_connections 2024-03-22T19:09:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.517+0000 7f3ceb200700 1 -- 172.21.15.86:0/1552004210 wait complete. 2024-03-22T19:09:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.518+0000 7f3ceb200700 1 Processor -- start 2024-03-22T19:09:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 -- start start 2024-03-22T19:09:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4100e60 0x7f3ce4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce4101820 0x7f3ce41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ce4106e30 0x7f3ce419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419e260 con 0x7f3ce4100e60 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419e3d0 con 0x7f3ce4106e30 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ceb200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419e590 con 0x7f3ce4101820 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.519+0000 7f3ce3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4100e60 0x7f3ce4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4100e60 0x7f3ce4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42858/0 (socket says 172.21.15.86:42858) 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 -- 172.21.15.86:0/184863471 learned_addr learned my addr 172.21.15.86:0/184863471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3400700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce4101820 0x7f3ce41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce9200700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ce4106e30 0x7f3ce419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 -- 172.21.15.86:0/184863471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce4101820 msgr2=0x7f3ce41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce4101820 0x7f3ce41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 -- 172.21.15.86:0/184863471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ce4106e30 msgr2=0x7f3ce419dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ce4106e30 0x7f3ce419dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 -- 172.21.15.86:0/184863471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40095f0 con 0x7f3ce4100e60 2024-03-22T19:09:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce9200700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ce4106e30 0x7f3ce419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce3e00700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4100e60 0x7f3ce4199360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4006010 tx=0x7f3cd400bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd401d070 con 0x7f3ce4100e60 2024-03-22T19:09:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd4004bd0 con 0x7f3ce4100e60 2024-03-22T19:09:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.520+0000 7f3ceb200700 1 -- 172.21.15.86:0/184863471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce419e810 con 0x7f3ce4100e60 2024-03-22T19:09:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.521+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400fc30 con 0x7f3ce4100e60 2024-03-22T19:09:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.521+0000 7f3ceb200700 1 -- 172.21.15.86:0/184863471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce41a60d0 con 0x7f3ce4100e60 2024-03-22T19:09:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.522+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3cbc000fc0 con 0x7f3ce4100e60 2024-03-22T19:09:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.524+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f3cd402b440 con 0x7f3ce4100e60 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.525+0000 7f3ce0c00700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3cc80a0ba0 0x7f3cc80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.525+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3cd40cd1d0 con 0x7f3ce4100e60 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.525+0000 7f3ce3400700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3cc80a0ba0 0x7f3cc80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.525+0000 7f3ce0c00700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] conn(0x7f3cc80aa370 0x7f3cc80ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.525+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 --> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cc80ace40 con 0x7f3cc80aa370 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.525+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cd40cd540 con 0x7f3ce4100e60 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.526+0000 7f3ce9200700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] conn(0x7f3cc80aa370 0x7f3cc80ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.526+0000 7f3ce3400700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3cc80a0ba0 0x7f3cc80a3060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ce419a8c0 tx=0x7f3cd00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.526+0000 7f3ce9200700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] conn(0x7f3cc80aa370 0x7f3cc80ac790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.527+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== osd.0 v2:172.21.15.86:6802/2783447556 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3cc80ace40 con 0x7f3cc80aa370 2024-03-22T19:09:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.544+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 --> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3cbc002d40 con 0x7f3cc80aa370 2024-03-22T19:09:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.544+0000 7f3ce0c00700 1 -- 172.21.15.86:0/184863471 <== osd.0 v2:172.21.15.86:6802/2783447556 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3cbc002d40 con 0x7f3cc80aa370 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.544+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] conn(0x7f3cc80aa370 msgr2=0x7f3cc80ac790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.544+0000 7f3cda000700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] conn(0x7f3cc80aa370 0x7f3cc80ac790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.544+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3cc80a0ba0 msgr2=0x7f3cc80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.544+0000 7f3cda000700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3cc80a0ba0 0x7f3cc80a3060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ce419a8c0 tx=0x7f3cd00073d0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4100e60 msgr2=0x7f3ce4199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3cda000700 1 --2- 172.21.15.86:0/184863471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce4100e60 0x7f3ce4199360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4006010 tx=0x7f3cd400bb00 comp rx=0 tx=0).stop 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3ce9200700 1 -- 172.21.15.86:0/184863471 reap_dead start 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 shutdown_connections 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 >> 172.21.15.86:0/184863471 conn(0x7f3ce4077ed0 msgr2=0x7f3ce410c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 shutdown_connections 2024-03-22T19:09:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.545+0000 7f3cda000700 1 -- 172.21.15.86:0/184863471 wait complete. 2024-03-22T19:09:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 -- 172.21.15.86:0/405442646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 msgr2=0x7f253c10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 --2- 172.21.15.86:0/405442646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c10da90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f253400b0a0 tx=0x7f253400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 -- 172.21.15.86:0/405442646 shutdown_connections 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 --2- 172.21.15.86:0/405442646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c10da90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 --2- 172.21.15.86:0/405442646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c100de0 0x7f253c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 --2- 172.21.15.86:0/405442646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c101790 0x7f253c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 -- 172.21.15.86:0/405442646 >> 172.21.15.86:0/405442646 conn(0x7f253c0fc6b0 msgr2=0x7f253c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 -- 172.21.15.86:0/405442646 shutdown_connections 2024-03-22T19:09:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.708+0000 7f2540c00700 1 -- 172.21.15.86:0/405442646 wait complete. 2024-03-22T19:09:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 Processor -- start 2024-03-22T19:09:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 -- start start 2024-03-22T19:09:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c100de0 0x7f253c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c101790 0x7f253c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a54c0 con 0x7f253c105ec0 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a5630 con 0x7f253c100de0 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.709+0000 7f2540c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a57f0 con 0x7f253c101790 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42872/0 (socket says 172.21.15.86:42872) 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 -- 172.21.15.86:0/1808381108 learned_addr learned my addr 172.21.15.86:0/1808381108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253b400700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c100de0 0x7f253c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253aa00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c101790 0x7f253c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 -- 172.21.15.86:0/1808381108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c101790 msgr2=0x7f253c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c101790 0x7f253c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 -- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c100de0 msgr2=0x7f253c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c100de0 0x7f253c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 -- 172.21.15.86:0/1808381108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c009520 con 0x7f253c105ec0 2024-03-22T19:09:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253aa00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c101790 0x7f253c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.710+0000 7f253be00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c073b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f253400b070 tx=0x7f25340078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.711+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400e040 con 0x7f253c105ec0 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.711+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2528009520 con 0x7f253c105ec0 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.711+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f253401d470 con 0x7f253c105ec0 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.711+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340052a0 con 0x7f253c105ec0 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.711+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2534009d00 con 0x7f253c105ec0 2024-03-22T19:09:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.712+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f253c110db0 con 0x7f253c105ec0 2024-03-22T19:09:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.714+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f253401d5e0 con 0x7f253c105ec0 2024-03-22T19:09:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.715+0000 7f2533e00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25200a0ba0 0x7f25200a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.715+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f25340cd9f0 con 0x7f253c105ec0 2024-03-22T19:09:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.715+0000 7f253b400700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25200a0ba0 0x7f25200a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.715+0000 7f2533e00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] conn(0x7f25200aa380 0x7f25200ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.715+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 --> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25200ace50 con 0x7f25200aa380 2024-03-22T19:09:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.715+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25340cdd60 con 0x7f253c105ec0 2024-03-22T19:09:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.716+0000 7f253aa00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] conn(0x7f25200aa380 0x7f25200ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.716+0000 7f253b400700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25200a0ba0 0x7f25200a3060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f252c009df0 tx=0x7f252c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.716+0000 7f253aa00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] conn(0x7f25200aa380 0x7f25200ac7a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.717+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== osd.1 v2:172.21.15.131:6800/3559403915 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f25200ace50 con 0x7f25200aa380 2024-03-22T19:09:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.733+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 --> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f253c1022c0 con 0x7f25200aa380 2024-03-22T19:09:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2533e00700 1 -- 172.21.15.86:0/1808381108 <== osd.1 v2:172.21.15.131:6800/3559403915 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f253c1022c0 con 0x7f25200aa380 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] conn(0x7f25200aa380 msgr2=0x7f25200ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] conn(0x7f25200aa380 0x7f25200ac7a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25200a0ba0 msgr2=0x7f25200a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25200a0ba0 0x7f25200a3060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f252c009df0 tx=0x7f252c009260 comp rx=0 tx=0).stop 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 msgr2=0x7f253c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 --2- 172.21.15.86:0/1808381108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f253c105ec0 0x7f253c073b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f253400b070 tx=0x7f25340078e0 comp rx=0 tx=0).stop 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f253be00700 1 -- 172.21.15.86:0/1808381108 reap_dead start 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 shutdown_connections 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 >> 172.21.15.86:0/1808381108 conn(0x7f253c0fc6b0 msgr2=0x7f253c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 shutdown_connections 2024-03-22T19:09:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:42.734+0000 7f2540c00700 1 -- 172.21.15.86:0/1808381108 wait complete. 2024-03-22T19:09:43.251 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:43 smithi086 ceph-mon[38032]: pgmap v246: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-22T19:09:43.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:43 smithi131 ceph-mon[78559]: pgmap v246: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-22T19:09:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:43 smithi167 ceph-mon[47728]: pgmap v246: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-22T19:09:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 -- 172.21.15.86:0/2415277634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 msgr2=0x7fb19410dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 --2- 172.21.15.86:0/2415277634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19410dc60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b0a0 tx=0x7fb18c00b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 -- 172.21.15.86:0/2415277634 shutdown_connections 2024-03-22T19:09:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 --2- 172.21.15.86:0/2415277634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19410dc60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 --2- 172.21.15.86:0/2415277634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb194100fb0 0x7fb194101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 --2- 172.21.15.86:0/2415277634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb194101960 0x7fb1941059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.908+0000 7fb199400700 1 -- 172.21.15.86:0/2415277634 >> 172.21.15.86:0/2415277634 conn(0x7fb1940fc800 msgr2=0x7fb1940fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.909+0000 7fb199400700 1 -- 172.21.15.86:0/2415277634 shutdown_connections 2024-03-22T19:09:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.909+0000 7fb199400700 1 -- 172.21.15.86:0/2415277634 wait complete. 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 Processor -- start 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 -- start start 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb194100fb0 0x7fb194197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb194101960 0x7fb194197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19419c1e0 con 0x7fb194106090 2024-03-22T19:09:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19419c350 con 0x7fb194100fb0 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb199400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19419c510 con 0x7fb194101960 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb193200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb193200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42910/0 (socket says 172.21.15.86:42910) 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb193200700 1 -- 172.21.15.86:0/2838775792 learned_addr learned my addr 172.21.15.86:0/2838775792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb191e00700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb194101960 0x7fb194197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.910+0000 7fb192800700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb194100fb0 0x7fb194197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb193200700 1 -- 172.21.15.86:0/2838775792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb194101960 msgr2=0x7fb194197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb193200700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb194101960 0x7fb194197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb193200700 1 -- 172.21.15.86:0/2838775792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb194100fb0 msgr2=0x7fb194197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb193200700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb194100fb0 0x7fb194197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb193200700 1 -- 172.21.15.86:0/2838775792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c009d00 con 0x7fb194106090 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb191e00700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb194101960 0x7fb194197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb192800700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb194100fb0 0x7fb194197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb193200700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19419bbd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b070 tx=0x7fb18c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00e040 con 0x7fb194106090 2024-03-22T19:09:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c01d470 con 0x7fb194106090 2024-03-22T19:09:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19419c790 con 0x7fb194106090 2024-03-22T19:09:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.911+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c005280 con 0x7fb194106090 2024-03-22T19:09:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.912+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1941a4050 con 0x7fb194106090 2024-03-22T19:09:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.912+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb194102490 con 0x7fb194106090 2024-03-22T19:09:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.915+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fb18c01d5e0 con 0x7fb194106090 2024-03-22T19:09:43.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.916+0000 7fb18b400700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1780a0ba0 0x7fb1780a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.916+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb18c0cd280 con 0x7fb194106090 2024-03-22T19:09:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.917+0000 7fb192800700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1780a0ba0 0x7fb1780a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.917+0000 7fb18b400700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] conn(0x7fb1780aa380 0x7fb1780ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.917+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 --> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1780ace50 con 0x7fb1780aa380 2024-03-22T19:09:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.917+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb18c0cd720 con 0x7fb194106090 2024-03-22T19:09:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.917+0000 7fb192800700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1780a0ba0 0x7fb1780a3060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb184011490 tx=0x7fb184009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.918+0000 7fb191e00700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] conn(0x7fb1780aa380 0x7fb1780ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.919+0000 7fb191e00700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] conn(0x7fb1780aa380 0x7fb1780ac7a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.920+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== osd.9 v2:172.21.15.86:6826/1269690910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb1780ace50 con 0x7fb1780aa380 2024-03-22T19:09:43.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 -- 172.21.15.86:0/3230734912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614100fb0 msgr2=0x7fc614101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 --2- 172.21.15.86:0/3230734912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614100fb0 0x7fc614101390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc604009990 tx=0x7fc604009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 -- 172.21.15.86:0/3230734912 shutdown_connections 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 --2- 172.21.15.86:0/3230734912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc614106090 0x7fc61410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 --2- 172.21.15.86:0/3230734912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614100fb0 0x7fc614101390 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 --2- 172.21.15.86:0/3230734912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614101960 0x7fc6141059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 -- 172.21.15.86:0/3230734912 >> 172.21.15.86:0/3230734912 conn(0x7fc6140fc800 msgr2=0x7fc6140fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 -- 172.21.15.86:0/3230734912 shutdown_connections 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.319+0000 7fc61a200700 1 -- 172.21.15.86:0/3230734912 wait complete. 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.320+0000 7fc61a200700 1 Processor -- start 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.320+0000 7fc61a200700 1 -- start start 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.320+0000 7fc61a200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc614100fb0 0x7fc614197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.320+0000 7fc61a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614101960 0x7fc614197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc61a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614106090 0x7fc61419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc61a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419c100 con 0x7fc614101960 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc61a200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419c270 con 0x7fc614106090 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc61a200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419c430 con 0x7fc614100fb0 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614101960 0x7fc614197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614101960 0x7fc614197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42884/0 (socket says 172.21.15.86:42884) 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 -- 172.21.15.86:0/3954991337 learned_addr learned my addr 172.21.15.86:0/3954991337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc613400700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc614100fb0 0x7fc614197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc613e00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614106090 0x7fc61419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 -- 172.21.15.86:0/3954991337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc614100fb0 msgr2=0x7fc614197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc614100fb0 0x7fc614197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 -- 172.21.15.86:0/3954991337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614106090 msgr2=0x7fc61419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614106090 0x7fc61419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 -- 172.21.15.86:0/3954991337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6040095f0 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc613e00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614106090 0x7fc61419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.321+0000 7fc612a00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614101960 0x7fc614197790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc60000d980 tx=0x7fc60000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.322+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600009930 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.322+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc600010460 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.322+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61419c710 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.322+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60000f620 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.322+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6141a3f70 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.322+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc614110f20 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.325+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc6000108d0 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.326+0000 7fc60be00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5f80a0ba0 0x7fc5f80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.326+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6000cbf40 con 0x7fc614101960 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.326+0000 7fc613400700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5f80a0ba0 0x7fc5f80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.326+0000 7fc60be00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] conn(0x7fc5f80aa380 0x7fc5f80ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.326+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 --> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f80ace50 con 0x7fc5f80aa380 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.326+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6000cc2b0 con 0x7fc614101960 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.327+0000 7fc613e00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] conn(0x7fc5f80aa380 0x7fc5f80ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.327+0000 7fc613400700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5f80a0ba0 0x7fc5f80a3060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098e0 tx=0x7fc604005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.327+0000 7fc613e00700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] conn(0x7fc5f80aa380 0x7fc5f80ac7a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.328+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== osd.3 v2:172.21.15.86:6810/1218616366 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc5f80ace50 con 0x7fc5f80aa380 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.345+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 --> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc614061430 con 0x7fc5f80aa380 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.345+0000 7fc60be00700 1 -- 172.21.15.86:0/3954991337 <== osd.3 v2:172.21.15.86:6810/1218616366 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc614061430 con 0x7fc5f80aa380 2024-03-22T19:09:43.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] conn(0x7fc5f80aa380 msgr2=0x7fc5f80ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] conn(0x7fc5f80aa380 0x7fc5f80ac7a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5f80a0ba0 msgr2=0x7fc5f80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5f80a0ba0 0x7fc5f80a3060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098e0 tx=0x7fc604005960 comp rx=0 tx=0).stop 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614101960 msgr2=0x7fc614197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 --2- 172.21.15.86:0/3954991337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc614101960 0x7fc614197790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc60000d980 tx=0x7fc60000dc90 comp rx=0 tx=0).stop 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc613e00700 1 -- 172.21.15.86:0/3954991337 reap_dead start 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 shutdown_connections 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 >> 172.21.15.86:0/3954991337 conn(0x7fc6140fc800 msgr2=0x7fc6140fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 shutdown_connections 2024-03-22T19:09:43.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.346+0000 7fc61a200700 1 -- 172.21.15.86:0/3954991337 wait complete. 2024-03-22T19:09:43.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.937+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 --> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb194110f60 con 0x7fb1780aa380 2024-03-22T19:09:43.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.938+0000 7fb18b400700 1 -- 172.21.15.86:0/2838775792 <== osd.9 v2:172.21.15.86:6826/1269690910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb194110f60 con 0x7fb1780aa380 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.938+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] conn(0x7fb1780aa380 msgr2=0x7fb1780ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.938+0000 7fb199400700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] conn(0x7fb1780aa380 0x7fb1780ac7a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.938+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1780a0ba0 msgr2=0x7fb1780a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1780a0ba0 0x7fb1780a3060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb184011490 tx=0x7fb184009260 comp rx=0 tx=0).stop 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 msgr2=0x7fb19419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 --2- 172.21.15.86:0/2838775792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb194106090 0x7fb19419bbd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b070 tx=0x7fb18c0078c0 comp rx=0 tx=0).stop 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb193200700 1 -- 172.21.15.86:0/2838775792 reap_dead start 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 shutdown_connections 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 >> 172.21.15.86:0/2838775792 conn(0x7fb1940fc800 msgr2=0x7fb1940feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 shutdown_connections 2024-03-22T19:09:43.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:43.939+0000 7fb199400700 1 -- 172.21.15.86:0/2838775792 wait complete. 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 -- 172.21.15.86:0/1750861934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974106090 msgr2=0x7fb97410dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 --2- 172.21.15.86:0/1750861934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974106090 0x7fb97410dc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00b0a0 tx=0x7fb96c00b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 -- 172.21.15.86:0/1750861934 shutdown_connections 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 --2- 172.21.15.86:0/1750861934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974106090 0x7fb97410dc60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 --2- 172.21.15.86:0/1750861934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb974100fb0 0x7fb974101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 --2- 172.21.15.86:0/1750861934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb974101960 0x7fb9741059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 -- 172.21.15.86:0/1750861934 >> 172.21.15.86:0/1750861934 conn(0x7fb9740fc800 msgr2=0x7fb9740fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 -- 172.21.15.86:0/1750861934 shutdown_connections 2024-03-22T19:09:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.392+0000 7fb979e00700 1 -- 172.21.15.86:0/1750861934 wait complete. 2024-03-22T19:09:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.393+0000 7fb979e00700 1 Processor -- start 2024-03-22T19:09:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.393+0000 7fb979e00700 1 -- start start 2024-03-22T19:09:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb979e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb974100fb0 0x7fb974197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb979e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974101960 0x7fb9741978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb979e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb974106090 0x7fb97419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb979e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419c220 con 0x7fb974101960 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb979e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419c390 con 0x7fb974100fb0 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb979e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419c550 con 0x7fb974106090 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974101960 0x7fb9741978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974101960 0x7fb9741978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42936/0 (socket says 172.21.15.86:42936) 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 -- 172.21.15.86:0/2583499674 learned_addr learned my addr 172.21.15.86:0/2583499674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb972a00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb974100fb0 0x7fb974197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb973400700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb974106090 0x7fb97419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 -- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb974106090 msgr2=0x7fb97419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb974106090 0x7fb97419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 -- 172.21.15.86:0/2583499674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb974100fb0 msgr2=0x7fb974197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb974100fb0 0x7fb974197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.394+0000 7fb96be00700 1 -- 172.21.15.86:0/2583499674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009d00 con 0x7fb974101960 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.395+0000 7fb96be00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974101960 0x7fb9741978b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e970 tx=0x7fb95c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.395+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c009790 con 0x7fb974101960 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.395+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb95c00eec0 con 0x7fb974101960 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.395+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97419c830 con 0x7fb974101960 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.395+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c005590 con 0x7fb974101960 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.395+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9741a4090 con 0x7fb974101960 2024-03-22T19:09:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.396+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb974110f60 con 0x7fb974101960 2024-03-22T19:09:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.398+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fb95c00cbd0 con 0x7fb974101960 2024-03-22T19:09:44.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.399+0000 7fb978a00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb9580a0ba0 0x7fb9580a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:44.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.399+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb95c014070 con 0x7fb974101960 2024-03-22T19:09:44.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.399+0000 7fb972a00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb9580a0ba0 0x7fb9580a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:44.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.400+0000 7fb978a00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] conn(0x7fb9580aa380 0x7fb9580ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:44.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.400+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 --> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9580ace50 con 0x7fb9580aa380 2024-03-22T19:09:44.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.400+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb95c0cc800 con 0x7fb974101960 2024-03-22T19:09:44.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.400+0000 7fb972a00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb9580a0ba0 0x7fb9580a3060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb964009c60 tx=0x7fb964009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:44.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.400+0000 7fb973400700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] conn(0x7fb9580aa380 0x7fb9580ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:44.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.401+0000 7fb973400700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] conn(0x7fb9580aa380 0x7fb9580ac7a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:44.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.402+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== osd.8 v2:172.21.15.167:6816/2277591889 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb9580ace50 con 0x7fb9580aa380 2024-03-22T19:09:44.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.418+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 --> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb974061430 con 0x7fb9580aa380 2024-03-22T19:09:44.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb978a00700 1 -- 172.21.15.86:0/2583499674 <== osd.8 v2:172.21.15.167:6816/2277591889 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb974061430 con 0x7fb9580aa380 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] conn(0x7fb9580aa380 msgr2=0x7fb9580ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] conn(0x7fb9580aa380 0x7fb9580ac7a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb9580a0ba0 msgr2=0x7fb9580a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb9580a0ba0 0x7fb9580a3060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb964009c60 tx=0x7fb964009260 comp rx=0 tx=0).stop 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974101960 msgr2=0x7fb9741978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 --2- 172.21.15.86:0/2583499674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb974101960 0x7fb9741978b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e970 tx=0x7fb95c00ec80 comp rx=0 tx=0).stop 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb973400700 1 -- 172.21.15.86:0/2583499674 reap_dead start 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 shutdown_connections 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 >> 172.21.15.86:0/2583499674 conn(0x7fb9740fc800 msgr2=0x7fb9740feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.419+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 shutdown_connections 2024-03-22T19:09:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:44.420+0000 7fb979e00700 1 -- 172.21.15.86:0/2583499674 wait complete. 2024-03-22T19:09:44.683 INFO:teuthology.orchestra.run.smithi086.stdout:210453397535 2024-03-22T19:09:44.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-22T19:09:45.067 INFO:teuthology.orchestra.run.smithi086.stdout:34359738424 2024-03-22T19:09:45.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-22T19:09:45.153 INFO:teuthology.orchestra.run.smithi086.stdout:322122547216 2024-03-22T19:09:45.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-22T19:09:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.173+0000 7f4794800700 1 -- 172.21.15.86:0/3239059446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073aa0 msgr2=0x7f478c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.173+0000 7f4794800700 1 --2- 172.21.15.86:0/3239059446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073aa0 0x7f478c10d0b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f477c009990 tx=0x7f477c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 -- 172.21.15.86:0/3239059446 shutdown_connections 2024-03-22T19:09:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 --2- 172.21.15.86:0/3239059446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f478c10d5f0 0x7f478c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 --2- 172.21.15.86:0/3239059446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073180 0x7f478c073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 --2- 172.21.15.86:0/3239059446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073aa0 0x7f478c10d0b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 -- 172.21.15.86:0/3239059446 >> 172.21.15.86:0/3239059446 conn(0x7f478c0fc800 msgr2=0x7f478c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 -- 172.21.15.86:0/3239059446 shutdown_connections 2024-03-22T19:09:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.174+0000 7f4794800700 1 -- 172.21.15.86:0/3239059446 wait complete. 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.175+0000 7f4794800700 1 Processor -- start 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.175+0000 7f4794800700 1 -- start start 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4794800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073180 0x7f478c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4794800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073aa0 0x7f478c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4794800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f478c10d5f0 0x7f478c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4794800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19e320 con 0x7f478c073180 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4794800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19e490 con 0x7f478c073aa0 2024-03-22T19:09:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4794800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19e650 con 0x7f478c10d5f0 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4791e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073180 0x7f478c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4791400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073aa0 0x7f478c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4792800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f478c10d5f0 0x7f478c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4791400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073aa0 0x7f478c1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:37688/0 (socket says 172.21.15.86:37688) 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.176+0000 7f4791400700 1 -- 172.21.15.86:0/2226682554 learned_addr learned my addr 172.21.15.86:0/2226682554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4791400700 1 -- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f478c10d5f0 msgr2=0x7f478c19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4791400700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f478c10d5f0 0x7f478c19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4791400700 1 -- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073180 msgr2=0x7f478c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4791400700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c073180 0x7f478c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4791400700 1 -- 172.21.15.86:0/2226682554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0095f0 con 0x7f478c073aa0 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4791400700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073aa0 0x7f478c1999b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00faf0 con 0x7f478c073aa0 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c19e8d0 con 0x7f478c073aa0 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477c02e070 con 0x7f478c073aa0 2024-03-22T19:09:45.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.177+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c033e00 con 0x7f478c073aa0 2024-03-22T19:09:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.178+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c06a9a0 con 0x7f478c073aa0 2024-03-22T19:09:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.179+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f478c1131e0 con 0x7f478c073aa0 2024-03-22T19:09:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.181+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f477c00fc50 con 0x7f478c073aa0 2024-03-22T19:09:45.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.182+0000 7f4786a00700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f47740a0b20 0x7f47740a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.182+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f477c0deee0 con 0x7f478c073aa0 2024-03-22T19:09:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.183+0000 7f4791e00700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f47740a0b20 0x7f47740a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.183+0000 7f4786a00700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] conn(0x7f47740aa300 0x7f47740ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.183+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 --> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47740acdd0 con 0x7f47740aa300 2024-03-22T19:09:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.183+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477c0df250 con 0x7f478c073aa0 2024-03-22T19:09:45.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.183+0000 7f4792800700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] conn(0x7f47740aa300 0x7f47740ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.183+0000 7f4791e00700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f47740a0b20 0x7f47740a2fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4780009c60 tx=0x7f4780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:45.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.184+0000 7f4792800700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] conn(0x7f47740aa300 0x7f47740ac720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:45.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.185+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== osd.5 v2:172.21.15.167:6808/634422158 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f47740acdd0 con 0x7f47740aa300 2024-03-22T19:09:45.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.201+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 --> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f478c0614b0 con 0x7f47740aa300 2024-03-22T19:09:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.202+0000 7f4786a00700 1 -- 172.21.15.86:0/2226682554 <== osd.5 v2:172.21.15.167:6808/634422158 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f478c0614b0 con 0x7f47740aa300 2024-03-22T19:09:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] conn(0x7f47740aa300 msgr2=0x7f47740ac720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] conn(0x7f47740aa300 0x7f47740ac720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f47740a0b20 msgr2=0x7f47740a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f47740a0b20 0x7f47740a2fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4780009c60 tx=0x7f4780009260 comp rx=0 tx=0).stop 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073aa0 msgr2=0x7f478c1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 --2- 172.21.15.86:0/2226682554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c073aa0 0x7f478c1999b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c00bf90 comp rx=0 tx=0).stop 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4792800700 1 -- 172.21.15.86:0/2226682554 reap_dead start 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 shutdown_connections 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 >> 172.21.15.86:0/2226682554 conn(0x7f478c0fc800 msgr2=0x7f478c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 shutdown_connections 2024-03-22T19:09:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.203+0000 7f4794800700 1 -- 172.21.15.86:0/2226682554 wait complete. 2024-03-22T19:09:45.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:45 smithi086 ceph-mon[38032]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:09:45.413 INFO:teuthology.orchestra.run.smithi086.stdout:270582939671 2024-03-22T19:09:45.414 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-22T19:09:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:45 smithi167 ceph-mon[47728]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:09:45.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.636+0000 7ff5f0800700 1 -- 172.21.15.86:0/967830819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8073b00 msgr2=0x7ff5e810d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.636+0000 7ff5f0800700 1 --2- 172.21.15.86:0/967830819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8073b00 0x7ff5e810d050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009990 tx=0x7ff5d8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 -- 172.21.15.86:0/967830819 shutdown_connections 2024-03-22T19:09:45.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 --2- 172.21.15.86:0/967830819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e810d590 0x7ff5e810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 --2- 172.21.15.86:0/967830819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e80731e0 0x7ff5e80735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 --2- 172.21.15.86:0/967830819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8073b00 0x7ff5e810d050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 -- 172.21.15.86:0/967830819 >> 172.21.15.86:0/967830819 conn(0x7ff5e80fc800 msgr2=0x7ff5e80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:45.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 -- 172.21.15.86:0/967830819 shutdown_connections 2024-03-22T19:09:45.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.637+0000 7ff5f0800700 1 -- 172.21.15.86:0/967830819 wait complete. 2024-03-22T19:09:45.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.638+0000 7ff5f0800700 1 Processor -- start 2024-03-22T19:09:45.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 -- start start 2024-03-22T19:09:45.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80731e0 0x7ff5e81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e8073b00 0x7ff5e8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e810d590 0x7ff5e819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5ede00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80731e0 0x7ff5e81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e819e2f0 con 0x7ff5e810d590 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e819e460 con 0x7ff5e8073b00 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5f0800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e819e620 con 0x7ff5e80731e0 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5ede00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80731e0 0x7ff5e81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:56482/0 (socket says 172.21.15.86:56482) 2024-03-22T19:09:45.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.639+0000 7ff5ede00700 1 -- 172.21.15.86:0/1707277626 learned_addr learned my addr 172.21.15.86:0/1707277626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:45.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ed400700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e8073b00 0x7ff5e8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ee800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e810d590 0x7ff5e819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ede00700 1 -- 172.21.15.86:0/1707277626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e8073b00 msgr2=0x7ff5e8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ede00700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e8073b00 0x7ff5e8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ede00700 1 -- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e810d590 msgr2=0x7ff5e819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ede00700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e810d590 0x7ff5e819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ede00700 1 -- 172.21.15.86:0/1707277626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d80095f0 con 0x7ff5e80731e0 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ee800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e810d590 0x7ff5e819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.640+0000 7ff5ede00700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80731e0 0x7ff5e81993f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e9a0 tx=0x7ff5dc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.641+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc009790 con 0x7ff5e80731e0 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.641+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5dc00cbd0 con 0x7ff5e80731e0 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.641+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc018680 con 0x7ff5e80731e0 2024-03-22T19:09:45.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.641+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e819e900 con 0x7ff5e80731e0 2024-03-22T19:09:45.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.641+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e81a6160 con 0x7ff5e80731e0 2024-03-22T19:09:45.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.642+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5e8074890 con 0x7ff5e80731e0 2024-03-22T19:09:45.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.644+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7ff5dc00cd40 con 0x7ff5e80731e0 2024-03-22T19:09:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.645+0000 7ff5e2a00700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5d00a0ba0 0x7ff5d00a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.646+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff5dc014070 con 0x7ff5e80731e0 2024-03-22T19:09:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.646+0000 7ff5ed400700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5d00a0ba0 0x7ff5d00a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.646+0000 7ff5e2a00700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] conn(0x7ff5d00aa380 0x7ff5d00ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.646+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 --> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5d00ace50 con 0x7ff5d00aa380 2024-03-22T19:09:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.646+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5dc0ce730 con 0x7ff5e80731e0 2024-03-22T19:09:45.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.646+0000 7ff5ee800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] conn(0x7ff5d00aa380 0x7ff5d00ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:45.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.647+0000 7ff5ed400700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5d00a0ba0 0x7ff5d00a3060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff5e819a950 tx=0x7ff5d800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:45.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.647+0000 7ff5ee800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] conn(0x7ff5d00aa380 0x7ff5d00ac7a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:45.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.648+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== osd.2 v2:172.21.15.167:6800/2732206899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff5d00ace50 con 0x7ff5d00aa380 2024-03-22T19:09:45.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.665+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 --> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5e81131c0 con 0x7ff5d00aa380 2024-03-22T19:09:45.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.666+0000 7ff5e2a00700 1 -- 172.21.15.86:0/1707277626 <== osd.2 v2:172.21.15.167:6800/2732206899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5e81131c0 con 0x7ff5d00aa380 2024-03-22T19:09:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] conn(0x7ff5d00aa380 msgr2=0x7ff5d00ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] conn(0x7ff5d00aa380 0x7ff5d00ac7a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5d00a0ba0 msgr2=0x7ff5d00a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5d00a0ba0 0x7ff5d00a3060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff5e819a950 tx=0x7ff5d800b260 comp rx=0 tx=0).stop 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80731e0 msgr2=0x7ff5e81993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 --2- 172.21.15.86:0/1707277626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80731e0 0x7ff5e81993f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e9a0 tx=0x7ff5dc00ecb0 comp rx=0 tx=0).stop 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5ee800700 1 -- 172.21.15.86:0/1707277626 reap_dead start 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 shutdown_connections 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 >> 172.21.15.86:0/1707277626 conn(0x7ff5e80fc800 msgr2=0x7ff5e80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 shutdown_connections 2024-03-22T19:09:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:45.667+0000 7ff5f0800700 1 -- 172.21.15.86:0/1707277626 wait complete. 2024-03-22T19:09:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:45 smithi131 ceph-mon[78559]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:09:45.863 INFO:teuthology.orchestra.run.smithi086.stdout:103079215149 2024-03-22T19:09:45.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-22T19:09:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 -- 172.21.15.86:0/3722370490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178101960 msgr2=0x7f41781059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 --2- 172.21.15.86:0/3722370490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178101960 0x7f41781059b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4160009990 tx=0x7f4160009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 -- 172.21.15.86:0/3722370490 shutdown_connections 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 --2- 172.21.15.86:0/3722370490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178106090 0x7f417810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 --2- 172.21.15.86:0/3722370490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4178100fb0 0x7f4178101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 --2- 172.21.15.86:0/3722370490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178101960 0x7f41781059b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 -- 172.21.15.86:0/3722370490 >> 172.21.15.86:0/3722370490 conn(0x7f41780fc800 msgr2=0x7f41780fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 -- 172.21.15.86:0/3722370490 shutdown_connections 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.289+0000 7f417f600700 1 -- 172.21.15.86:0/3722370490 wait complete. 2024-03-22T19:09:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.290+0000 7f417f600700 1 Processor -- start 2024-03-22T19:09:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.290+0000 7f417f600700 1 -- start start 2024-03-22T19:09:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4178100fb0 0x7f41780730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417f600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178101960 0x7f41780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417f600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178106090 0x7f4178073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781a5550 con 0x7f4178100fb0 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417f600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781a56c0 con 0x7f4178101960 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417f600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781a5880 con 0x7f4178106090 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f417cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4178100fb0 0x7f41780730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f416fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178101960 0x7f41780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.291+0000 7f416fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178101960 0x7f41780735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:37710/0 (socket says 172.21.15.86:37710) 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f416fe00700 1 -- 172.21.15.86:0/1830755618 learned_addr learned my addr 172.21.15.86:0/1830755618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f417d600700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178106090 0x7f4178073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f416fe00700 1 -- 172.21.15.86:0/1830755618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178106090 msgr2=0x7f4178073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f416fe00700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4178106090 0x7f4178073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f416fe00700 1 -- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4178100fb0 msgr2=0x7f41780730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f416fe00700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4178100fb0 0x7f41780730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.292+0000 7f416fe00700 1 -- 172.21.15.86:0/1830755618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41600095f0 con 0x7f4178101960 2024-03-22T19:09:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.310+0000 7f416fe00700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178101960 0x7f41780735f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.317+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416001d070 con 0x7f4178101960 2024-03-22T19:09:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.317+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4160022470 con 0x7f4178101960 2024-03-22T19:09:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.317+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160017650 con 0x7f4178101960 2024-03-22T19:09:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.317+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41781a5b00 con 0x7f4178101960 2024-03-22T19:09:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.317+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41781a5ec0 con 0x7f4178101960 2024-03-22T19:09:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.318+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4178110f00 con 0x7f4178101960 2024-03-22T19:09:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.320+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f41600225e0 con 0x7f4178101960 2024-03-22T19:09:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.321+0000 7f4176000700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41580a0ba0 0x7f41580a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.321+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f41600cd9f0 con 0x7f4178101960 2024-03-22T19:09:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.322+0000 7f417cc00700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41580a0ba0 0x7f41580a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.322+0000 7f4176000700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] conn(0x7f41580aa380 0x7f41580ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.322+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 --> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41580ace50 con 0x7f41580aa380 2024-03-22T19:09:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.322+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41600cdd60 con 0x7f4178101960 2024-03-22T19:09:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.322+0000 7f417cc00700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41580a0ba0 0x7f41580a3060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4168009c60 tx=0x7f4168009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.322+0000 7f417d600700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] conn(0x7f41580aa380 0x7f41580ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.323+0000 7f417d600700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] conn(0x7f41580aa380 0x7f41580ac7a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:46.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.324+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== osd.10 v2:172.21.15.131:6824/407606440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f41580ace50 con 0x7f41580aa380 2024-03-22T19:09:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.340+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 --> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41780614b0 con 0x7f41580aa380 2024-03-22T19:09:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f4176000700 1 -- 172.21.15.86:0/1830755618 <== osd.10 v2:172.21.15.131:6824/407606440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f41780614b0 con 0x7f41580aa380 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] conn(0x7f41580aa380 msgr2=0x7f41580ac7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] conn(0x7f41580aa380 0x7f41580ac7a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41580a0ba0 msgr2=0x7f41580a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41580a0ba0 0x7f41580a3060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4168009c60 tx=0x7f4168009260 comp rx=0 tx=0).stop 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178101960 msgr2=0x7f41780735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 --2- 172.21.15.86:0/1830755618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4178101960 0x7f41780735f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160004730 comp rx=0 tx=0).stop 2024-03-22T19:09:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417d600700 1 -- 172.21.15.86:0/1830755618 reap_dead start 2024-03-22T19:09:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 shutdown_connections 2024-03-22T19:09:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 >> 172.21.15.86:0/1830755618 conn(0x7f41780fc800 msgr2=0x7f41780fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 shutdown_connections 2024-03-22T19:09:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:46.341+0000 7f417f600700 1 -- 172.21.15.86:0/1830755618 wait complete. 2024-03-22T19:09:46.436 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:46.585 INFO:teuthology.orchestra.run.smithi086.stdout:55834574902 2024-03-22T19:09:46.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-22T19:09:47.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:47 smithi086 ceph-mon[38032]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-22T19:09:47.393 INFO:teuthology.orchestra.run.smithi086.stdout:77309411379 2024-03-22T19:09:47.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-22T19:09:47.409 INFO:teuthology.orchestra.run.smithi086.stdout:240518168604 2024-03-22T19:09:47.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-22T19:09:47.418 INFO:teuthology.orchestra.run.smithi086.stdout:158913789991 2024-03-22T19:09:47.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-22T19:09:47.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:47 smithi131 ceph-mon[78559]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-22T19:09:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:47 smithi167 ceph-mon[47728]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-22T19:09:48.495 INFO:teuthology.orchestra.run.smithi086.stdout:292057776147 2024-03-22T19:09:48.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-22T19:09:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:49.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:49 smithi131 ceph-mon[78559]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-22T19:09:49.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:49 smithi086 ceph-mon[38032]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-22T19:09:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:49 smithi167 ceph-mon[47728]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 -- 172.21.15.86:0/1789025678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b20101570 msgr2=0x7f5b201019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 --2- 172.21.15.86:0/1789025678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b20101570 0x7f5b201019f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009990 tx=0x7f5b0c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 -- 172.21.15.86:0/1789025678 shutdown_connections 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 --2- 172.21.15.86:0/1789025678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b20101f30 0x7f5b2010ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 --2- 172.21.15.86:0/1789025678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b200ffe20 0x7f5b20100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 --2- 172.21.15.86:0/1789025678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b20101570 0x7f5b201019f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 -- 172.21.15.86:0/1789025678 >> 172.21.15.86:0/1789025678 conn(0x7f5b200fb5a0 msgr2=0x7f5b200fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 -- 172.21.15.86:0/1789025678 shutdown_connections 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.812+0000 7f5b25c00700 1 -- 172.21.15.86:0/1789025678 wait complete. 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 Processor -- start 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 -- start start 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 0x7f5b20197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b20101570 0x7f5b201975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b20101f30 0x7f5b2019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019bf70 con 0x7f5b200ffe20 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019c0e0 con 0x7f5b20101570 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.813+0000 7f5b25c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019c2a0 con 0x7f5b20101f30 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 0x7f5b20197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 0x7f5b20197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48126/0 (socket says 172.21.15.86:48126) 2024-03-22T19:09:49.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 -- 172.21.15.86:0/2185083040 learned_addr learned my addr 172.21.15.86:0/2185083040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1f400700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b20101570 0x7f5b201975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b25200700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b20101f30 0x7f5b2019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 -- 172.21.15.86:0/2185083040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b20101f30 msgr2=0x7f5b2019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b20101f30 0x7f5b2019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 -- 172.21.15.86:0/2185083040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b20101570 msgr2=0x7f5b201975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b20101570 0x7f5b201975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 -- 172.21.15.86:0/2185083040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0095f0 con 0x7f5b200ffe20 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1fe00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 0x7f5b20197070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000e9a0 tx=0x7f5b1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.814+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10009790 con 0x7f5b200ffe20 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.815+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2019c580 con 0x7f5b200ffe20 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.815+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b10010430 con 0x7f5b200ffe20 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.815+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10018680 con 0x7f5b200ffe20 2024-03-22T19:09:49.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.815+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a3de0 con 0x7f5b200ffe20 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.816+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20109110 con 0x7f5b200ffe20 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.818+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f5b100105a0 con 0x7f5b200ffe20 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.819+0000 7f5b1cc00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5b040a0b20 0x7f5b040a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.819+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b10014070 con 0x7f5b200ffe20 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.819+0000 7f5b1f400700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5b040a0b20 0x7f5b040a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.820+0000 7f5b1f400700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5b040a0b20 0x7f5b040a2fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b201985d0 tx=0x7f5b0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:49.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:49.823+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5b100529f0 con 0x7f5b200ffe20 2024-03-22T19:09:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.050+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5b20198380 con 0x7f5b200ffe20 2024-03-22T19:09:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.051+0000 7f5b1cc00700 1 -- 172.21.15.86:0/2185083040 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b10095bf0 con 0x7f5b200ffe20 2024-03-22T19:09:50.053 INFO:teuthology.orchestra.run.smithi086.stdout:34359738425 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5b040a0b20 msgr2=0x7f5b040a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5b040a0b20 0x7f5b040a2fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b201985d0 tx=0x7f5b0c005fd0 comp rx=0 tx=0).stop 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 msgr2=0x7f5b20197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 0x7f5b20197070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000e9a0 tx=0x7f5b1000ecb0 comp rx=0 tx=0).stop 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 shutdown_connections 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5b040a0b20 0x7f5b040a2fe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b20101f30 0x7f5b2019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b200ffe20 0x7f5b20197070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 --2- 172.21.15.86:0/2185083040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b20101570 0x7f5b201975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.053+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 >> 172.21.15.86:0/2185083040 conn(0x7f5b200fb5a0 msgr2=0x7f5b20068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:50.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.054+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 shutdown_connections 2024-03-22T19:09:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:50.054+0000 7f5b25c00700 1 -- 172.21.15.86:0/2185083040 wait complete. 2024-03-22T19:09:50.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2185083040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T19:09:50.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-22T19:09:50.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-03-22T19:09:50.483 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-22T19:09:50.483 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-22T19:09:50.483 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-22T19:09:50.483 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:50 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-22T19:09:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2185083040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T19:09:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-22T19:09:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-03-22T19:09:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-22T19:09:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-22T19:09:50.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-22T19:09:50.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:50 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-22T19:09:50.696 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2185083040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T19:09:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-22T19:09:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-03-22T19:09:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-22T19:09:50.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-22T19:09:50.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-22T19:09:50.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:50 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-22T19:09:50.769 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738424 got 34359738425 for osd.0 2024-03-22T19:09:50.769 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 -- 172.21.15.86:0/1667810112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040101960 msgr2=0x7f50401059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 --2- 172.21.15.86:0/1667810112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040101960 0x7f50401059b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f502c009990 tx=0x7f502c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 -- 172.21.15.86:0/1667810112 shutdown_connections 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 --2- 172.21.15.86:0/1667810112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040106090 0x7f504010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 --2- 172.21.15.86:0/1667810112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040100fb0 0x7f5040101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 --2- 172.21.15.86:0/1667810112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040101960 0x7f50401059b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 -- 172.21.15.86:0/1667810112 >> 172.21.15.86:0/1667810112 conn(0x7f50400fc800 msgr2=0x7f50400fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 -- 172.21.15.86:0/1667810112 shutdown_connections 2024-03-22T19:09:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.013+0000 7f5045400700 1 -- 172.21.15.86:0/1667810112 wait complete. 2024-03-22T19:09:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 Processor -- start 2024-03-22T19:09:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 -- start start 2024-03-22T19:09:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040100fb0 0x7f50401993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 0x7f5040199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040106090 0x7f504019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504019e2d0 con 0x7f5040101960 2024-03-22T19:09:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504019e440 con 0x7f5040100fb0 2024-03-22T19:09:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.015+0000 7f5045400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504019e600 con 0x7f5040106090 2024-03-22T19:09:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 0x7f5040199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 0x7f5040199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48140/0 (socket says 172.21.15.86:48140) 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 -- 172.21.15.86:0/1082895529 learned_addr learned my addr 172.21.15.86:0/1082895529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503e800700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040100fb0 0x7f50401993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503f200700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040106090 0x7f504019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 -- 172.21.15.86:0/1082895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040106090 msgr2=0x7f504019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040106090 0x7f504019dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 -- 172.21.15.86:0/1082895529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040100fb0 msgr2=0x7f50401993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040100fb0 0x7f50401993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503de00700 1 -- 172.21.15.86:0/1082895529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c0095f0 con 0x7f5040101960 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.016+0000 7f503e800700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040100fb0 0x7f50401993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.017+0000 7f503de00700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 0x7f5040199910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f502c000c00 tx=0x7f502c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.017+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c01d070 con 0x7f5040101960 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.017+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502c021410 con 0x7f5040101960 2024-03-22T19:09:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.017+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504019e880 con 0x7f5040101960 2024-03-22T19:09:51.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.017+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50401a6140 con 0x7f5040101960 2024-03-22T19:09:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.017+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c017650 con 0x7f5040101960 2024-03-22T19:09:51.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.018+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040102490 con 0x7f5040101960 2024-03-22T19:09:51.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.020+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f502c021580 con 0x7f5040101960 2024-03-22T19:09:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.022+0000 7f5037400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50240a0b20 0x7f50240a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.022+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f502c0cd3b0 con 0x7f5040101960 2024-03-22T19:09:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.022+0000 7f503e800700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50240a0b20 0x7f50240a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.022+0000 7f503e800700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50240a0b20 0x7f50240a2fe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5030011490 tx=0x7f5030009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:51.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.025+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f502c096ca0 con 0x7f5040101960 2024-03-22T19:09:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.265+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f504019a6e0 con 0x7f5040101960 2024-03-22T19:09:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.265+0000 7f5037400700 1 -- 172.21.15.86:0/1082895529 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f502c09a2c0 con 0x7f5040101960 2024-03-22T19:09:51.267 INFO:teuthology.orchestra.run.smithi086.stdout:103079215150 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50240a0b20 msgr2=0x7f50240a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50240a0b20 0x7f50240a2fe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5030011490 tx=0x7f5030009260 comp rx=0 tx=0).stop 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 msgr2=0x7f5040199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 0x7f5040199910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f502c000c00 tx=0x7f502c004730 comp rx=0 tx=0).stop 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 shutdown_connections 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50240a0b20 0x7f50240a2fe0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5040106090 0x7f504019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5040100fb0 0x7f50401993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 --2- 172.21.15.86:0/1082895529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5040101960 0x7f5040199910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 >> 172.21.15.86:0/1082895529 conn(0x7f50400fc800 msgr2=0x7f50400feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 shutdown_connections 2024-03-22T19:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.269+0000 7f5045400700 1 -- 172.21.15.86:0/1082895529 wait complete. 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 -- 172.21.15.86:0/368820058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 msgr2=0x7f85f81059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 --2- 172.21.15.86:0/368820058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f81059b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009990 tx=0x7f85e4009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 -- 172.21.15.86:0/368820058 shutdown_connections 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 --2- 172.21.15.86:0/368820058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f8106090 0x7f85f810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 --2- 172.21.15.86:0/368820058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f8100fb0 0x7f85f8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 --2- 172.21.15.86:0/368820058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f81059b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 -- 172.21.15.86:0/368820058 >> 172.21.15.86:0/368820058 conn(0x7f85f80fc800 msgr2=0x7f85f80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 -- 172.21.15.86:0/368820058 shutdown_connections 2024-03-22T19:09:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.320+0000 7f85fd400700 1 -- 172.21.15.86:0/368820058 wait complete. 2024-03-22T19:09:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.321+0000 7f85fd400700 1 Processor -- start 2024-03-22T19:09:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 -- start start 2024-03-22T19:09:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f8100fb0 0x7f85f8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f8106090 0x7f85f819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f819e320 con 0x7f85f8101960 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f819e490 con 0x7f85f8100fb0 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85fd400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f819e650 con 0x7f85f8106090 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85f5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85f5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f8199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48154/0 (socket says 172.21.15.86:48154) 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85f5e00700 1 -- 172.21.15.86:0/2531037837 learned_addr learned my addr 172.21.15.86:0/2531037837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85f6800700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f8100fb0 0x7f85f8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.322+0000 7f85f7200700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f8106090 0x7f85f819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85f5e00700 1 -- 172.21.15.86:0/2531037837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f8106090 msgr2=0x7f85f819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85f5e00700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f8106090 0x7f85f819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85f5e00700 1 -- 172.21.15.86:0/2531037837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f8100fb0 msgr2=0x7f85f8199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85f5e00700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f8100fb0 0x7f85f8199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85f5e00700 1 -- 172.21.15.86:0/2531037837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e40095f0 con 0x7f85f8101960 2024-03-22T19:09:51.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85f5e00700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f8199960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f85e4000c00 tx=0x7f85e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e401d070 con 0x7f85f8101960 2024-03-22T19:09:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e4022470 con 0x7f85f8101960 2024-03-22T19:09:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4017650 con 0x7f85f8101960 2024-03-22T19:09:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f819e8d0 con 0x7f85f8101960 2024-03-22T19:09:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.323+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f81a5fc0 con 0x7f85f8101960 2024-03-22T19:09:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.325+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f8102490 con 0x7f85f8101960 2024-03-22T19:09:51.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.326+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f85e400fad0 con 0x7f85f8101960 2024-03-22T19:09:51.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.328+0000 7f85ef400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85dc0a0b20 0x7f85dc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.328+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f85e40cd660 con 0x7f85f8101960 2024-03-22T19:09:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.329+0000 7f85f6800700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85dc0a0b20 0x7f85dc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:51.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.329+0000 7f85f6800700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85dc0a0b20 0x7f85dc0a2fe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85e8007be0 tx=0x7f85e80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:51.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.332+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f85e4097070 con 0x7f85f8101960 2024-03-22T19:09:51.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-22T19:09:51.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:51 smithi131 ceph-mon[78559]: osdmap e81: 12 total, 12 up, 12 in 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-22T19:09:51.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:51 smithi086 ceph-mon[38032]: osdmap e81: 12 total, 12 up, 12 in 2024-03-22T19:09:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.549+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f85f81a6150 con 0x7f85f8101960 2024-03-22T19:09:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.550+0000 7f85ef400700 1 -- 172.21.15.86:0/2531037837 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85e4027020 con 0x7f85f8101960 2024-03-22T19:09:51.551 INFO:teuthology.orchestra.run.smithi086.stdout:137438953516 2024-03-22T19:09:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85dc0a0b20 msgr2=0x7f85dc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85dc0a0b20 0x7f85dc0a2fe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85e8007be0 tx=0x7f85e80073d0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 msgr2=0x7f85f8199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f8199960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f85e4000c00 tx=0x7f85e4005740 comp rx=0 tx=0).stop 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 shutdown_connections 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85dc0a0b20 0x7f85dc0a2fe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f8106090 0x7f85f819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f8100fb0 0x7f85f8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 --2- 172.21.15.86:0/2531037837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f8101960 0x7f85f8199960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 >> 172.21.15.86:0/2531037837 conn(0x7f85f80fc800 msgr2=0x7f85f80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 shutdown_connections 2024-03-22T19:09:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:51.552+0000 7f85fd400700 1 -- 172.21.15.86:0/2531037837 wait complete. 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-22T19:09:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:51 smithi167 ceph-mon[47728]: osdmap e81: 12 total, 12 up, 12 in 2024-03-22T19:09:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:52.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953513 got 137438953516 for osd.4 2024-03-22T19:09:52.180 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:52.369 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215149 got 103079215150 for osd.3 2024-03-22T19:09:52.369 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.390+0000 7f6818600700 1 -- 172.21.15.86:0/3058199248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810106ec0 msgr2=0x7f68101072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.390+0000 7f6818600700 1 --2- 172.21.15.86:0/3058199248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810106ec0 0x7f68101072a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6804009990 tx=0x7f6804009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 -- 172.21.15.86:0/3058199248 shutdown_connections 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 --2- 172.21.15.86:0/3058199248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810069140 0x7f681010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 --2- 172.21.15.86:0/3058199248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810106ec0 0x7f68101072a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 --2- 172.21.15.86:0/3058199248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6810068780 0x7f6810068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 -- 172.21.15.86:0/3058199248 >> 172.21.15.86:0/3058199248 conn(0x7f6810076db0 msgr2=0x7f68100771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 -- 172.21.15.86:0/3058199248 shutdown_connections 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.391+0000 7f6818600700 1 -- 172.21.15.86:0/3058199248 wait complete. 2024-03-22T19:09:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.392+0000 7f6818600700 1 Processor -- start 2024-03-22T19:09:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.392+0000 7f6818600700 1 -- start start 2024-03-22T19:09:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6818600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 0x7f68101992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6818600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6810069140 0x7f6810199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6818600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810106ec0 0x7f681019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6818600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681019e1f0 con 0x7f6810068780 2024-03-22T19:09:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6818600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681019e360 con 0x7f6810069140 2024-03-22T19:09:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6818600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681019e520 con 0x7f6810106ec0 2024-03-22T19:09:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6815c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 0x7f68101992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6815c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 0x7f68101992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48174/0 (socket says 172.21.15.86:48174) 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6815c00700 1 -- 172.21.15.86:0/164939099 learned_addr learned my addr 172.21.15.86:0/164939099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.393+0000 7f6815200700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6810069140 0x7f6810199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6816600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810106ec0 0x7f681019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6815c00700 1 -- 172.21.15.86:0/164939099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810106ec0 msgr2=0x7f681019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6815c00700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810106ec0 0x7f681019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6815c00700 1 -- 172.21.15.86:0/164939099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6810069140 msgr2=0x7f6810199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6815c00700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6810069140 0x7f6810199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6815c00700 1 -- 172.21.15.86:0/164939099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040095f0 con 0x7f6810068780 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f6815c00700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 0x7f68101992f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68040095c0 tx=0x7f680400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.394+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680401d070 con 0x7f6810068780 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.395+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6804005030 con 0x7f6810068780 2024-03-22T19:09:52.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.395+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804004a90 con 0x7f6810068780 2024-03-22T19:09:52.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.395+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681019e7a0 con 0x7f6810068780 2024-03-22T19:09:52.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.395+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68101a6060 con 0x7f6810068780 2024-03-22T19:09:52.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.396+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810102470 con 0x7f6810068780 2024-03-22T19:09:52.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.398+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f6804005410 con 0x7f6810068780 2024-03-22T19:09:52.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.399+0000 7f680e000700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67f80a0ba0 0x7f67f80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:52.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.399+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f68040ccef0 con 0x7f6810068780 2024-03-22T19:09:52.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.399+0000 7f6815200700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67f80a0ba0 0x7f67f80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:52.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.400+0000 7f6815200700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67f80a0ba0 0x7f67f80a3060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f681019a850 tx=0x7f67fc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:52.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.403+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f68040967e0 con 0x7f6810068780 2024-03-22T19:09:52.467 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:52.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:52 smithi086 ceph-mon[38032]: osdmap e82: 12 total, 12 up, 12 in 2024-03-22T19:09:52.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:52 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1082895529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T19:09:52.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:52 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2531037837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T19:09:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:52 smithi167 ceph-mon[47728]: osdmap e82: 12 total, 12 up, 12 in 2024-03-22T19:09:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:52 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1082895529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T19:09:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:52 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2531037837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T19:09:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.618+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f681019a600 con 0x7f6810068780 2024-03-22T19:09:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.618+0000 7f680e000700 1 -- 172.21.15.86:0/164939099 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6804026090 con 0x7f6810068780 2024-03-22T19:09:52.620 INFO:teuthology.orchestra.run.smithi086.stdout:322122547219 2024-03-22T19:09:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67f80a0ba0 msgr2=0x7f67f80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67f80a0ba0 0x7f67f80a3060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f681019a850 tx=0x7f67fc0073d0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 msgr2=0x7f68101992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 0x7f68101992f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68040095c0 tx=0x7f680400f690 comp rx=0 tx=0).stop 2024-03-22T19:09:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 shutdown_connections 2024-03-22T19:09:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f67f80a0ba0 0x7f67f80a3060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810106ec0 0x7f681019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6810068780 0x7f68101992f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 --2- 172.21.15.86:0/164939099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6810069140 0x7f6810199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 >> 172.21.15.86:0/164939099 conn(0x7f6810076db0 msgr2=0x7f68100fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 shutdown_connections 2024-03-22T19:09:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:52.621+0000 7f6818600700 1 -- 172.21.15.86:0/164939099 wait complete. 2024-03-22T19:09:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:52 smithi131 ceph-mon[78559]: osdmap e82: 12 total, 12 up, 12 in 2024-03-22T19:09:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:52 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1082895529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T19:09:52.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:52 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2531037837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T19:09:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:53 smithi086 ceph-mon[38032]: pgmap v253: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-22T19:09:53.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:53 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/164939099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-22T19:09:53.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547216 got 322122547219 for osd.11 2024-03-22T19:09:53.567 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:53 smithi167 ceph-mon[47728]: pgmap v253: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-22T19:09:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:53 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/164939099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-22T19:09:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:53 smithi131 ceph-mon[78559]: pgmap v253: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-22T19:09:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:53 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/164939099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-22T19:09:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.084+0000 7fe4fde00700 1 -- 172.21.15.86:0/4267048696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073150 msgr2=0x7fe4f8073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.084+0000 7fe4fde00700 1 --2- 172.21.15.86:0/4267048696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073150 0x7fe4f8073530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009990 tx=0x7fe4e8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.085+0000 7fe4fde00700 1 -- 172.21.15.86:0/4267048696 shutdown_connections 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.085+0000 7fe4fde00700 1 --2- 172.21.15.86:0/4267048696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8111d60 0x7fe4f8114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.085+0000 7fe4fde00700 1 --2- 172.21.15.86:0/4267048696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073150 0x7fe4f8073530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.085+0000 7fe4fde00700 1 --2- 172.21.15.86:0/4267048696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f8073a70 0x7fe4f8111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.085+0000 7fe4fde00700 1 -- 172.21.15.86:0/4267048696 >> 172.21.15.86:0/4267048696 conn(0x7fe4f80fc800 msgr2=0x7fe4f80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.085+0000 7fe4fde00700 1 -- 172.21.15.86:0/4267048696 shutdown_connections 2024-03-22T19:09:54.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.086+0000 7fe4fde00700 1 -- 172.21.15.86:0/4267048696 wait complete. 2024-03-22T19:09:54.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.087+0000 7fe4fde00700 1 Processor -- start 2024-03-22T19:09:54.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.087+0000 7fe4fde00700 1 -- start start 2024-03-22T19:09:54.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4fde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 0x7fe4f81a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4fde00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073a70 0x7fe4f81a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4fde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f8111d60 0x7fe4f819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4fde00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f819dcf0 con 0x7fe4f8073150 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4fde00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f819de60 con 0x7fe4f8111d60 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4fde00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f819e020 con 0x7fe4f8073a70 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 0x7fe4f81a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 0x7fe4f81a3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48190/0 (socket says 172.21.15.86:48190) 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 -- 172.21.15.86:0/3593365566 learned_addr learned my addr 172.21.15.86:0/3593365566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6000700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073a70 0x7fe4f81a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f7400700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f8111d60 0x7fe4f819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 -- 172.21.15.86:0/3593365566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073a70 msgr2=0x7fe4f81a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073a70 0x7fe4f81a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 -- 172.21.15.86:0/3593365566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f8111d60 msgr2=0x7fe4f819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f8111d60 0x7fe4f819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.088+0000 7fe4f6a00700 1 -- 172.21.15.86:0/3593365566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80095f0 con 0x7fe4f8073150 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4f6000700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073a70 0x7fe4f81a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:09:54.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4f6a00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 0x7fe4f81a3550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8000c00 tx=0x7fe4e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:54.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e801d070 con 0x7fe4f8073150 2024-03-22T19:09:54.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4e8022470 con 0x7fe4f8073150 2024-03-22T19:09:54.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8017650 con 0x7fe4f8073150 2024-03-22T19:09:54.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f819e2a0 con 0x7fe4f8073150 2024-03-22T19:09:54.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.089+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f81aa540 con 0x7fe4f8073150 2024-03-22T19:09:54.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.090+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f8104240 con 0x7fe4f8073150 2024-03-22T19:09:54.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.092+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fe4e80225e0 con 0x7fe4f8073150 2024-03-22T19:09:54.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.094+0000 7fe4fca00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe4dc0a0ba0 0x7fe4dc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.094+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe4e80cd3e0 con 0x7fe4f8073150 2024-03-22T19:09:54.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.094+0000 7fe4f6000700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe4dc0a0ba0 0x7fe4dc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.095+0000 7fe4f6000700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe4dc0a0ba0 0x7fe4dc0a3060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe4f819eda0 tx=0x7fe4e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:54.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.097+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe4e805d9f0 con 0x7fe4f8073150 2024-03-22T19:09:54.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.325+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe4f8074890 con 0x7fe4f8073150 2024-03-22T19:09:54.328 INFO:teuthology.orchestra.run.smithi086.stdout:210453397537 2024-03-22T19:09:54.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.325+0000 7fe4fca00700 1 -- 172.21.15.86:0/3593365566 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4e8096cd0 con 0x7fe4f8073150 2024-03-22T19:09:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe4dc0a0ba0 msgr2=0x7fe4dc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe4dc0a0ba0 0x7fe4dc0a3060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe4f819eda0 tx=0x7fe4e4009260 comp rx=0 tx=0).stop 2024-03-22T19:09:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 msgr2=0x7fe4f81a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 0x7fe4f81a3550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8000c00 tx=0x7fe4e8005740 comp rx=0 tx=0).stop 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 shutdown_connections 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe4dc0a0ba0 0x7fe4dc0a3060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f8111d60 0x7fe4f819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8073150 0x7fe4f81a3550 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 --2- 172.21.15.86:0/3593365566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f8073a70 0x7fe4f81a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 >> 172.21.15.86:0/3593365566 conn(0x7fe4f80fc800 msgr2=0x7fe4f80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 shutdown_connections 2024-03-22T19:09:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.328+0000 7fe4fde00700 1 -- 172.21.15.86:0/3593365566 wait complete. 2024-03-22T19:09:54.578 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 -- 172.21.15.86:0/2148070255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3148073b00 msgr2=0x7f314810d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 --2- 172.21.15.86:0/2148070255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3148073b00 0x7f314810d050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3134009990 tx=0x7f3134009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 -- 172.21.15.86:0/2148070255 shutdown_connections 2024-03-22T19:09:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 --2- 172.21.15.86:0/2148070255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f314810d590 0x7f314810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 --2- 172.21.15.86:0/2148070255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31480731e0 0x7f31480735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 --2- 172.21.15.86:0/2148070255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3148073b00 0x7f314810d050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 -- 172.21.15.86:0/2148070255 >> 172.21.15.86:0/2148070255 conn(0x7f31480fc800 msgr2=0x7f31480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 -- 172.21.15.86:0/2148070255 shutdown_connections 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.464+0000 7f314ee00700 1 -- 172.21.15.86:0/2148070255 wait complete. 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.465+0000 7f314ee00700 1 Processor -- start 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 -- start start 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31480731e0 0x7f31481993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3148073b00 0x7f3148199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 0x7f314819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314819e2f0 con 0x7f314810d590 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314819e460 con 0x7f3148073b00 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ee00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314819e620 con 0x7f31480731e0 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 0x7f314819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f3147e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31480731e0 0x7f31481993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 0x7f314819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48206/0 (socket says 172.21.15.86:48206) 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f314ce00700 1 -- 172.21.15.86:0/1225759965 learned_addr learned my addr 172.21.15.86:0/1225759965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.466+0000 7f3147400700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3148073b00 0x7f3148199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3147e00700 1 -- 172.21.15.86:0/1225759965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3148073b00 msgr2=0x7f3148199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3147e00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3148073b00 0x7f3148199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3147e00700 1 -- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 msgr2=0x7f314819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3147e00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 0x7f314819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3147e00700 1 -- 172.21.15.86:0/1225759965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340095f0 con 0x7f31480731e0 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f314ce00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 0x7f314819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3147e00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31480731e0 0x7f31481993f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f313800e9f0 tx=0x7f313800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800c9c0 con 0x7f31480731e0 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314819e900 con 0x7f31480731e0 2024-03-22T19:09:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3138010430 con 0x7f31480731e0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.467+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138018680 con 0x7f31480731e0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.468+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31481a6160 con 0x7f31480731e0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.469+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31481087a0 con 0x7f31480731e0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.471+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f3138010860 con 0x7f31480731e0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.472+0000 7f3144c00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f312c0a0b20 0x7f312c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.473+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3138014070 con 0x7f31480731e0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.473+0000 7f3147400700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f312c0a0b20 0x7f312c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.473+0000 7f3147400700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f312c0a0b20 0x7f312c0a2fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f314819a950 tx=0x7f3134005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:54.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.474+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3138095e40 con 0x7f31480731e0 2024-03-22T19:09:54.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.691+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f314819eaf0 con 0x7f31480731e0 2024-03-22T19:09:54.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.692+0000 7f3144c00700 1 -- 172.21.15.86:0/1225759965 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3138099460 con 0x7f31480731e0 2024-03-22T19:09:54.693 INFO:teuthology.orchestra.run.smithi086.stdout:158913789993 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f312c0a0b20 msgr2=0x7f312c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f312c0a0b20 0x7f312c0a2fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f314819a950 tx=0x7f3134005fd0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31480731e0 msgr2=0x7f31481993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31480731e0 0x7f31481993f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f313800e9f0 tx=0x7f313800ed00 comp rx=0 tx=0).stop 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 shutdown_connections 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f312c0a0b20 0x7f312c0a2fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314810d590 0x7f314819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31480731e0 0x7f31481993f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.694+0000 7f314ee00700 1 --2- 172.21.15.86:0/1225759965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3148073b00 0x7f3148199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:54.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.695+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 >> 172.21.15.86:0/1225759965 conn(0x7f31480fc800 msgr2=0x7f31480fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:54.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.695+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 shutdown_connections 2024-03-22T19:09:54.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:54.695+0000 7f314ee00700 1 -- 172.21.15.86:0/1225759965 wait complete. 2024-03-22T19:09:55.078 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:55.180 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:55.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:55 smithi086 ceph-mon[38032]: pgmap v254: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-22T19:09:55.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:55 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3593365566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-22T19:09:55.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:55 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1225759965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T19:09:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:55 smithi167 ceph-mon[47728]: pgmap v254: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-22T19:09:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:55 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3593365566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-22T19:09:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:55 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1225759965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T19:09:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:55 smithi131 ceph-mon[78559]: pgmap v254: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-22T19:09:55.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:55 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3593365566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-22T19:09:55.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:55 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1225759965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T19:09:55.944 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:09:56.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789991 got 158913789993 for osd.5 2024-03-22T19:09:56.127 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:56.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397535 got 210453397537 for osd.7 2024-03-22T19:09:56.316 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:57.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:57 smithi086 ceph-mon[38032]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-03-22T19:09:57.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:57 smithi131 ceph-mon[78559]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-03-22T19:09:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:57 smithi167 ceph-mon[47728]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-03-22T19:09:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 -- 172.21.15.86:0/3941656218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0108280 msgr2=0x7f2ad010fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 --2- 172.21.15.86:0/3941656218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0108280 0x7f2ad010fe50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b0a0 tx=0x7f2ac800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 -- 172.21.15.86:0/3941656218 shutdown_connections 2024-03-22T19:09:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 --2- 172.21.15.86:0/3941656218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0108280 0x7f2ad010fe50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 --2- 172.21.15.86:0/3941656218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad01031a0 0x7f2ad0103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 --2- 172.21.15.86:0/3941656218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad0103b50 0x7f2ad0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 -- 172.21.15.86:0/3941656218 >> 172.21.15.86:0/3941656218 conn(0x7f2ad00fea50 msgr2=0x7f2ad0100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 -- 172.21.15.86:0/3941656218 shutdown_connections 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.717+0000 7f2ad6800700 1 -- 172.21.15.86:0/3941656218 wait complete. 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.718+0000 7f2ad6800700 1 Processor -- start 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.718+0000 7f2ad6800700 1 -- start start 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ad6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad01031a0 0x7f2ad01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ad6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 0x7f2ad0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ad6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0108280 0x7f2ad019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ad6800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad019e470 con 0x7f2ad0103b50 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ad6800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad019e5e0 con 0x7f2ad0108280 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ad6800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad019e7a0 con 0x7f2ad01031a0 2024-03-22T19:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 0x7f2ad0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2acf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad01031a0 0x7f2ad01995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 0x7f2ad0199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48232/0 (socket says 172.21.15.86:48232) 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 -- 172.21.15.86:0/216574758 learned_addr learned my addr 172.21.15.86:0/216574758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2acfe00700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0108280 0x7f2ad019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 -- 172.21.15.86:0/216574758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad01031a0 msgr2=0x7f2ad01995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad01031a0 0x7f2ad01995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 -- 172.21.15.86:0/216574758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0108280 msgr2=0x7f2ad019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0108280 0x7f2ad019de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2ac7e00700 1 -- 172.21.15.86:0/216574758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac8009d00 con 0x7f2ad0103b50 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.719+0000 7f2acf400700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad01031a0 0x7f2ad01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.720+0000 7f2ac7e00700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 0x7f2ad0199b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800e970 tx=0x7f2ab800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.720+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8009790 con 0x7f2ad0103b50 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.720+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab800c7e0 con 0x7f2ad0103b50 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.720+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad019ea80 con 0x7f2ad0103b50 2024-03-22T19:09:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.720+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80054e0 con 0x7f2ad0103b50 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.720+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a62e0 con 0x7f2ad0103b50 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.721+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad010b540 con 0x7f2ad0103b50 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.723+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f2ab800c950 con 0x7f2ad0103b50 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.724+0000 7f2accc00700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2ab40a0ba0 0x7f2ab40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.724+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f2ab8014070 con 0x7f2ad0103b50 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.724+0000 7f2acf400700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2ab40a0ba0 0x7f2ab40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.725+0000 7f2acf400700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2ab40a0ba0 0x7f2ab40a3060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0011490 tx=0x7f2ac0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.728+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2ab8095bc0 con 0x7f2ad0103b50 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 -- 172.21.15.86:0/1539200651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 msgr2=0x7f2ea40735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 --2- 172.21.15.86:0/1539200651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 0x7f2ea40735c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009990 tx=0x7f2e98009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 -- 172.21.15.86:0/1539200651 shutdown_connections 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 --2- 172.21.15.86:0/1539200651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea410d590 0x7f2ea410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 --2- 172.21.15.86:0/1539200651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 0x7f2ea40735c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 --2- 172.21.15.86:0/1539200651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea4073b00 0x7f2ea410d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 -- 172.21.15.86:0/1539200651 >> 172.21.15.86:0/1539200651 conn(0x7f2ea40fc800 msgr2=0x7f2ea40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:57.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 -- 172.21.15.86:0/1539200651 shutdown_connections 2024-03-22T19:09:57.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.904+0000 7f2eac600700 1 -- 172.21.15.86:0/1539200651 wait complete. 2024-03-22T19:09:57.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.905+0000 7f2eac600700 1 Processor -- start 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 -- start start 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 0x7f2ea4100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea4073b00 0x7f2ea41011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea410d590 0x7f2ea4101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41a9aa0 con 0x7f2ea4073b00 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41a9be0 con 0x7f2ea40731e0 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eac600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41a9da0 con 0x7f2ea410d590 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2ea9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea4073b00 0x7f2ea41011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2ea9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea4073b00 0x7f2ea41011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48248/0 (socket says 172.21.15.86:48248) 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2ea9200700 1 -- 172.21.15.86:0/705684713 learned_addr learned my addr 172.21.15.86:0/705684713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2ea9c00700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 0x7f2ea4100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.906+0000 7f2eaa600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea410d590 0x7f2ea4101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eaa600700 1 -- 172.21.15.86:0/705684713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 msgr2=0x7f2ea4100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eaa600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 0x7f2ea4100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eaa600700 1 -- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea4073b00 msgr2=0x7f2ea41011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eaa600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea4073b00 0x7f2ea41011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eaa600700 1 -- 172.21.15.86:0/705684713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980095f0 con 0x7f2ea410d590 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eaa600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea410d590 0x7f2ea4101730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009f60 tx=0x7f2e9c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c018070 con 0x7f2ea410d590 2024-03-22T19:09:57.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea41aa020 con 0x7f2ea410d590 2024-03-22T19:09:57.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e9c00f040 con 0x7f2ea410d590 2024-03-22T19:09:57.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea41aa520 con 0x7f2ea410d590 2024-03-22T19:09:57.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.907+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00ccf0 con 0x7f2ea410d590 2024-03-22T19:09:57.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.908+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e840052f0 con 0x7f2ea410d590 2024-03-22T19:09:57.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.910+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f2e9c004ad0 con 0x7f2ea410d590 2024-03-22T19:09:57.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.913+0000 7f2ea2000700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2e8c0a0ba0 0x7f2e8c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:57.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.913+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f2e9c0cc440 con 0x7f2ea410d590 2024-03-22T19:09:57.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.913+0000 7f2ea9c00700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2e8c0a0ba0 0x7f2e8c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:57.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.913+0000 7f2ea9c00700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2e8c0a0ba0 0x7f2e8c0a3060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800b2e0 tx=0x7f2e98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:57.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.914+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2e9c095d30 con 0x7f2ea410d590 2024-03-22T19:09:57.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.957+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f2ad0104680 con 0x7f2ad0103b50 2024-03-22T19:09:57.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.958+0000 7f2accc00700 1 -- 172.21.15.86:0/216574758 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ab80991e0 con 0x7f2ad0103b50 2024-03-22T19:09:57.959 INFO:teuthology.orchestra.run.smithi086.stdout:240518168607 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.961+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2ab40a0ba0 msgr2=0x7f2ab40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.961+0000 7f2ad6800700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2ab40a0ba0 0x7f2ab40a3060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0011490 tx=0x7f2ac0009260 comp rx=0 tx=0).stop 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.961+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 msgr2=0x7f2ad0199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.961+0000 7f2ad6800700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 0x7f2ad0199b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800e970 tx=0x7f2ab800ec80 comp rx=0 tx=0).stop 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 shutdown_connections 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2ab40a0ba0 0x7f2ab40a3060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0108280 0x7f2ad019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ad01031a0 0x7f2ad01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 --2- 172.21.15.86:0/216574758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad0103b50 0x7f2ad0199b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:57.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 >> 172.21.15.86:0/216574758 conn(0x7f2ad00fea50 msgr2=0x7f2ad00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:57.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 shutdown_connections 2024-03-22T19:09:57.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:57.962+0000 7f2ad6800700 1 -- 172.21.15.86:0/216574758 wait complete. 2024-03-22T19:09:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.150+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2e84005c90 con 0x7f2ea410d590 2024-03-22T19:09:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.151+0000 7f2ea2000700 1 -- 172.21.15.86:0/705684713 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e9c099350 con 0x7f2ea410d590 2024-03-22T19:09:58.152 INFO:teuthology.orchestra.run.smithi086.stdout:55834574905 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.153+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2e8c0a0ba0 msgr2=0x7f2e8c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.153+0000 7f2eac600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2e8c0a0ba0 0x7f2e8c0a3060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800b2e0 tx=0x7f2e98005fd0 comp rx=0 tx=0).stop 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.153+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea410d590 msgr2=0x7f2ea4101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.153+0000 7f2eac600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea410d590 0x7f2ea4101730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009f60 tx=0x7f2e9c009f90 comp rx=0 tx=0).stop 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 shutdown_connections 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f2e8c0a0ba0 0x7f2e8c0a3060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ea410d590 0x7f2ea4101730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea40731e0 0x7f2ea4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 --2- 172.21.15.86:0/705684713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea4073b00 0x7f2ea41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 >> 172.21.15.86:0/705684713 conn(0x7f2ea40fc800 msgr2=0x7f2ea40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 shutdown_connections 2024-03-22T19:09:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.154+0000 7f2eac600700 1 -- 172.21.15.86:0/705684713 wait complete. 2024-03-22T19:09:58.249 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:58 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/216574758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-22T19:09:58.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:58 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/216574758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-22T19:09:58.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:58 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/216574758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-22T19:09:59.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574902 got 55834574905 for osd.1 2024-03-22T19:09:59.024 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:59.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168604 got 240518168607 for osd.8 2024-03-22T19:09:59.237 DEBUG:teuthology.parallel:result is None 2024-03-22T19:09:59.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:59 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/705684713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T19:09:59.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:59 smithi086 ceph-mon[38032]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 13 B/s, 1 objects/s recovering 2024-03-22T19:09:59.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:09:59 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/964656116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-22T19:09:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.711+0000 7f17e6400700 1 -- 172.21.15.86:0/1516810486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0100fb0 msgr2=0x7f17e0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.711+0000 7f17e6400700 1 --2- 172.21.15.86:0/1516810486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0100fb0 0x7f17e0101390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009990 tx=0x7f17d0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 -- 172.21.15.86:0/1516810486 shutdown_connections 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 --2- 172.21.15.86:0/1516810486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e0106090 0x7f17e010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 --2- 172.21.15.86:0/1516810486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0100fb0 0x7f17e0101390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 --2- 172.21.15.86:0/1516810486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e0101960 0x7f17e01059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 -- 172.21.15.86:0/1516810486 >> 172.21.15.86:0/1516810486 conn(0x7f17e00fc800 msgr2=0x7f17e00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 -- 172.21.15.86:0/1516810486 shutdown_connections 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.712+0000 7f17e6400700 1 -- 172.21.15.86:0/1516810486 wait complete. 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 Processor -- start 2024-03-22T19:09:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 -- start start 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e0100fb0 0x7f17e0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 0x7f17e0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e0106090 0x7f17e019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e019e2e0 con 0x7f17e0101960 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e019e450 con 0x7f17e0106090 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17e6400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e019e610 con 0x7f17e0100fb0 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.713+0000 7f17dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 0x7f17e0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 0x7f17e0199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50334/0 (socket says 172.21.15.86:50334) 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 -- 172.21.15.86:0/964656116 learned_addr learned my addr 172.21.15.86:0/964656116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17df400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e0100fb0 0x7f17e0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dfe00700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e0106090 0x7f17e019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 -- 172.21.15.86:0/964656116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e0100fb0 msgr2=0x7f17e0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e0100fb0 0x7f17e0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 -- 172.21.15.86:0/964656116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e0106090 msgr2=0x7f17e019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e0106090 0x7f17e019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 -- 172.21.15.86:0/964656116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00095f0 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17dea00700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 0x7f17e0199970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d980 tx=0x7f17cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00f890 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17cc010460 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e019e8f0 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.714+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00e610 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.715+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e01a6150 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.716+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0102490 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.718+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f17cc0105d0 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.720+0000 7f17d7e00700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17c40a0b20 0x7f17c40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.720+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f17cc0cbbd0 con 0x7f17e0101960 2024-03-22T19:09:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.720+0000 7f17df400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17c40a0b20 0x7f17c40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.721+0000 7f17df400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17c40a0b20 0x7f17c40a2fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17d00098e0 tx=0x7f17d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.724+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f17cc095430 con 0x7f17e0101960 2024-03-22T19:09:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.936+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f17e019a6f0 con 0x7f17e0101960 2024-03-22T19:09:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.936+0000 7f17d7e00700 1 -- 172.21.15.86:0/964656116 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17cc098a50 con 0x7f17e0101960 2024-03-22T19:09:59.266 INFO:teuthology.orchestra.run.smithi086.stdout:188978561062 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17c40a0b20 msgr2=0x7f17c40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17c40a0b20 0x7f17c40a2fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17d00098e0 tx=0x7f17d0005960 comp rx=0 tx=0).stop 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 msgr2=0x7f17e0199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 0x7f17e0199970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d980 tx=0x7f17cc00dc90 comp rx=0 tx=0).stop 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 shutdown_connections 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17c40a0b20 0x7f17c40a2fe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e0106090 0x7f17e019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e0100fb0 0x7f17e0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 --2- 172.21.15.86:0/964656116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e0101960 0x7f17e0199970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.938+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 >> 172.21.15.86:0/964656116 conn(0x7f17e00fc800 msgr2=0x7f17e00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.939+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 shutdown_connections 2024-03-22T19:09:59.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:58.939+0000 7f17e6400700 1 -- 172.21.15.86:0/964656116 wait complete. 2024-03-22T19:09:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:59 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/705684713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T19:09:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:59 smithi131 ceph-mon[78559]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 13 B/s, 1 objects/s recovering 2024-03-22T19:09:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:09:59 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/964656116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 -- 172.21.15.86:0/2706623771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934100fb0 msgr2=0x7f6934101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 --2- 172.21.15.86:0/2706623771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934100fb0 0x7f6934101390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6924009990 tx=0x7f6924009ca0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 -- 172.21.15.86:0/2706623771 shutdown_connections 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 --2- 172.21.15.86:0/2706623771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 0x7f693410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 --2- 172.21.15.86:0/2706623771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934100fb0 0x7f6934101390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 --2- 172.21.15.86:0/2706623771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934101960 0x7f69341059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 -- 172.21.15.86:0/2706623771 >> 172.21.15.86:0/2706623771 conn(0x7f69340fc800 msgr2=0x7f69340fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 -- 172.21.15.86:0/2706623771 shutdown_connections 2024-03-22T19:09:59.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.576+0000 7f6939e00700 1 -- 172.21.15.86:0/2706623771 wait complete. 2024-03-22T19:09:59.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:59 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/705684713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T19:09:59.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:59 smithi167 ceph-mon[47728]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 13 B/s, 1 objects/s recovering 2024-03-22T19:09:59.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:09:59 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/964656116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.577+0000 7f6939e00700 1 Processor -- start 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.577+0000 7f6939e00700 1 -- start start 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6939e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934100fb0 0x7f6934199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6939e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 0x7f69341999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6939e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 0x7f693419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6939e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419e2e0 con 0x7f6934101960 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6939e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419e450 con 0x7f6934100fb0 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6939e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419e610 con 0x7f6934106090 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6932000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 0x7f69341999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6933400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 0x7f693419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6933400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 0x7f693419dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:53218/0 (socket says 172.21.15.86:53218) 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6932000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 0x7f69341999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50350/0 (socket says 172.21.15.86:50350) 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6933400700 1 -- 172.21.15.86:0/505801739 learned_addr learned my addr 172.21.15.86:0/505801739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.578+0000 7f6932a00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934100fb0 0x7f6934199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932000700 1 -- 172.21.15.86:0/505801739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 msgr2=0x7f693419dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932000700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 0x7f693419dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932000700 1 -- 172.21.15.86:0/505801739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934100fb0 msgr2=0x7f6934199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932000700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934100fb0 0x7f6934199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932000700 1 -- 172.21.15.86:0/505801739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240095f0 con 0x7f6934101960 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932a00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934100fb0 0x7f6934199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6932000700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 0x7f69341999c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f693406a480 tx=0x7f692000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920009930 con 0x7f6934101960 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6920010460 con 0x7f6934101960 2024-03-22T19:09:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693419e8f0 con 0x7f6934101960 2024-03-22T19:09:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692000b5d0 con 0x7f6934101960 2024-03-22T19:09:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.579+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69341a6150 con 0x7f6934101960 2024-03-22T19:09:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.580+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934102490 con 0x7f6934101960 2024-03-22T19:09:59.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.583+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f6920010890 con 0x7f6934101960 2024-03-22T19:09:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.584+0000 7f6938a00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69180a0c70 0x7f69180a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:09:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.584+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f69200cbb90 con 0x7f6934101960 2024-03-22T19:09:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.585+0000 7f6932a00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69180a0c70 0x7f69180a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:09:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.585+0000 7f6932a00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69180a0c70 0x7f69180a3130 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f692400b5c0 tx=0x7f692401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:09:59.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.592+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f69200954f0 con 0x7f6934101960 2024-03-22T19:09:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.808+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f69341a62e0 con 0x7f6934101960 2024-03-22T19:09:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.809+0000 7f6938a00700 1 -- 172.21.15.86:0/505801739 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6920017090 con 0x7f6934101960 2024-03-22T19:09:59.811 INFO:teuthology.orchestra.run.smithi086.stdout:270582939674 2024-03-22T19:09:59.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69180a0c70 msgr2=0x7f69180a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69180a0c70 0x7f69180a3130 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f692400b5c0 tx=0x7f692401a040 comp rx=0 tx=0).stop 2024-03-22T19:09:59.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 msgr2=0x7f69341999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:09:59.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 0x7f69341999c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f693406a480 tx=0x7f692000dc30 comp rx=0 tx=0).stop 2024-03-22T19:09:59.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 shutdown_connections 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69180a0c70 0x7f69180a3130 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6934106090 0x7f693419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6934100fb0 0x7f6934199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 --2- 172.21.15.86:0/505801739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6934101960 0x7f69341999c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 >> 172.21.15.86:0/505801739 conn(0x7f69340fc800 msgr2=0x7f69340fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.811+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 shutdown_connections 2024-03-22T19:09:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:09:59.812+0000 7f6939e00700 1 -- 172.21.15.86:0/505801739 wait complete. 2024-03-22T19:10:00.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561059 got 188978561062 for osd.6 2024-03-22T19:10:00.112 DEBUG:teuthology.parallel:result is None 2024-03-22T19:10:00.456 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:00 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/505801739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-22T19:10:00.457 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:00 smithi086 ceph-mon[38032]: overall HEALTH_OK 2024-03-22T19:10:00.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:00 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/505801739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-22T19:10:00.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:00 smithi131 ceph-mon[78559]: overall HEALTH_OK 2024-03-22T19:10:00.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939671 got 270582939674 for osd.9 2024-03-22T19:10:00.479 DEBUG:teuthology.parallel:result is None 2024-03-22T19:10:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:00 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/505801739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-22T19:10:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:00 smithi167 ceph-mon[47728]: overall HEALTH_OK 2024-03-22T19:10:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 -- 172.21.15.86:0/3608318979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4073a70 msgr2=0x7f20a4111820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3608318979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4073a70 0x7f20a4111820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2090009990 tx=0x7f2090009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 -- 172.21.15.86:0/3608318979 shutdown_connections 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3608318979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4111d60 0x7f20a4114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3608318979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20a4073150 0x7f20a4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3608318979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4073a70 0x7f20a4111820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.762+0000 7f20a9a00700 1 -- 172.21.15.86:0/3608318979 >> 172.21.15.86:0/3608318979 conn(0x7f20a40fc800 msgr2=0x7f20a40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.763+0000 7f20a9a00700 1 -- 172.21.15.86:0/3608318979 shutdown_connections 2024-03-22T19:10:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.763+0000 7f20a9a00700 1 -- 172.21.15.86:0/3608318979 wait complete. 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 Processor -- start 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 -- start start 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20a4073150 0x7f20a41a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4073a70 0x7f20a41a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 0x7f20a419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419dcc0 con 0x7f20a4111d60 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419de30 con 0x7f20a4073150 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a9a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419dff0 con 0x7f20a4073a70 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 0x7f20a419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 0x7f20a419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50362/0 (socket says 172.21.15.86:50362) 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a3400700 1 -- 172.21.15.86:0/3050524321 learned_addr learned my addr 172.21.15.86:0/3050524321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a2000700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4073a70 0x7f20a41a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.764+0000 7f20a2a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20a4073150 0x7f20a41a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a3400700 1 -- 172.21.15.86:0/3050524321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4073a70 msgr2=0x7f20a41a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a3400700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4073a70 0x7f20a41a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a3400700 1 -- 172.21.15.86:0/3050524321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20a4073150 msgr2=0x7f20a41a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a3400700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20a4073150 0x7f20a41a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a3400700 1 -- 172.21.15.86:0/3050524321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900095f0 con 0x7f20a4111d60 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a3400700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 0x7f20a419d660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f209c00ef90 tx=0x7f209c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c018070 con 0x7f20a4111d60 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f209c00f040 con 0x7f20a4111d60 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a419e270 con 0x7f20a4111d60 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c003e00 con 0x7f20a4111d60 2024-03-22T19:10:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.765+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a41aa510 con 0x7f20a4111d60 2024-03-22T19:10:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.766+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a4104240 con 0x7f20a4111d60 2024-03-22T19:10:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.768+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f209c00ca50 con 0x7f20a4111d60 2024-03-22T19:10:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.770+0000 7f209b400700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20880a0ba0 0x7f20880a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.770+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f209c0cbd70 con 0x7f20a4111d60 2024-03-22T19:10:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.770+0000 7f20a2a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20880a0ba0 0x7f20880a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.770+0000 7f20a2a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20880a0ba0 0x7f20880a3060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2094005950 tx=0x7f20940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.773+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f209c095660 con 0x7f20a4111d60 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 -- 172.21.15.86:0/580786133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9604073b00 msgr2=0x7f960410ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 --2- 172.21.15.86:0/580786133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9604073b00 0x7f960410ce80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009990 tx=0x7f95f0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 -- 172.21.15.86:0/580786133 shutdown_connections 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 --2- 172.21.15.86:0/580786133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f960410d3c0 0x7f960410f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 --2- 172.21.15.86:0/580786133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 0x7f96040735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 --2- 172.21.15.86:0/580786133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9604073b00 0x7f960410ce80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 -- 172.21.15.86:0/580786133 >> 172.21.15.86:0/580786133 conn(0x7f96040fc6b0 msgr2=0x7f96040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:00.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 -- 172.21.15.86:0/580786133 shutdown_connections 2024-03-22T19:10:00.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.920+0000 7f9608a00700 1 -- 172.21.15.86:0/580786133 wait complete. 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.921+0000 7f9608a00700 1 Processor -- start 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 -- start start 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 0x7f9604199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9604073b00 0x7f96041998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 0x7f960419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960419e1d0 con 0x7f960410d3c0 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960419e340 con 0x7f96040731e0 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9608a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960419e500 con 0x7f9604073b00 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9603e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 0x7f960419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9603e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 0x7f960419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50372/0 (socket says 172.21.15.86:50372) 2024-03-22T19:10:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9603e00700 1 -- 172.21.15.86:0/3307494659 learned_addr learned my addr 172.21.15.86:0/3307494659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9603400700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 0x7f9604199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.922+0000 7f9602a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9604073b00 0x7f96041998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603400700 1 -- 172.21.15.86:0/3307494659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9604073b00 msgr2=0x7f96041998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603400700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9604073b00 0x7f96041998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603400700 1 -- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 msgr2=0x7f960419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603400700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 0x7f960419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603400700 1 -- 172.21.15.86:0/3307494659 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00095f0 con 0x7f96040731e0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603e00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 0x7f960419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9603400700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 0x7f9604199370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95f400e9f0 tx=0x7f95f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004220 con 0x7f96040731e0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95f4004cf0 con 0x7f96040731e0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960419e7e0 con 0x7f96040731e0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.923+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f400ce70 con 0x7f96040731e0 2024-03-22T19:10:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.924+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96041a6040 con 0x7f96040731e0 2024-03-22T19:10:00.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.924+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96041085d0 con 0x7f96040731e0 2024-03-22T19:10:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.927+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f95f4004750 con 0x7f96040731e0 2024-03-22T19:10:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.928+0000 7f95fbe00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95e80a0ba0 0x7f95e80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.928+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f95f4014070 con 0x7f96040731e0 2024-03-22T19:10:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.928+0000 7f9602a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95e80a0ba0 0x7f95e80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.929+0000 7f9602a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95e80a0ba0 0x7f95e80a3060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f960419a830 tx=0x7f95f000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.930+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f95f40959c0 con 0x7f96040731e0 2024-03-22T19:10:00.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.988+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20a419eb20 con 0x7f20a4111d60 2024-03-22T19:10:00.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.989+0000 7f209b400700 1 -- 172.21.15.86:0/3050524321 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f209c098c80 con 0x7f20a4111d60 2024-03-22T19:10:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:77309411382 2024-03-22T19:10:00.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20880a0ba0 msgr2=0x7f20880a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20880a0ba0 0x7f20880a3060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2094005950 tx=0x7f20940058e0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 msgr2=0x7f20a419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:00.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 0x7f20a419d660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f209c00ef90 tx=0x7f209c00c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 shutdown_connections 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f20880a0ba0 0x7f20880a3060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a4111d60 0x7f20a419d660 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20a4073150 0x7f20a41a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 --2- 172.21.15.86:0/3050524321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4073a70 0x7f20a41a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 >> 172.21.15.86:0/3050524321 conn(0x7f20a40fc800 msgr2=0x7f20a40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 shutdown_connections 2024-03-22T19:10:00.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:00.991+0000 7f20a9a00700 1 -- 172.21.15.86:0/3050524321 wait complete. 2024-03-22T19:10:01.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.145+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f9604074890 con 0x7f96040731e0 2024-03-22T19:10:01.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.146+0000 7f95fbe00700 1 -- 172.21.15.86:0/3307494659 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f95f4004e60 con 0x7f96040731e0 2024-03-22T19:10:01.147 INFO:teuthology.orchestra.run.smithi086.stdout:292057776150 2024-03-22T19:10:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95e80a0ba0 msgr2=0x7f95e80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95e80a0ba0 0x7f95e80a3060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f960419a830 tx=0x7f95f000b260 comp rx=0 tx=0).stop 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 msgr2=0x7f9604199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 0x7f9604199370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95f400e9f0 tx=0x7f95f400ed00 comp rx=0 tx=0).stop 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 shutdown_connections 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95e80a0ba0 0x7f95e80a3060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f960410d3c0 0x7f960419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040731e0 0x7f9604199370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 --2- 172.21.15.86:0/3307494659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9604073b00 0x7f96041998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 >> 172.21.15.86:0/3307494659 conn(0x7f96040fc6b0 msgr2=0x7f96040fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 shutdown_connections 2024-03-22T19:10:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:01.149+0000 7f9608a00700 1 -- 172.21.15.86:0/3307494659 wait complete. 2024-03-22T19:10:01.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:01 smithi086 ceph-mon[38032]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-03-22T19:10:01.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:01 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3050524321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T19:10:01.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411379 got 77309411382 for osd.2 2024-03-22T19:10:01.396 DEBUG:teuthology.parallel:result is None 2024-03-22T19:10:01.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:01 smithi131 ceph-mon[78559]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-03-22T19:10:01.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:01 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3050524321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T19:10:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:01 smithi167 ceph-mon[47728]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-03-22T19:10:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:01 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3050524321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T19:10:01.694 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776147 got 292057776150 for osd.10 2024-03-22T19:10:01.694 DEBUG:teuthology.parallel:result is None 2024-03-22T19:10:01.694 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-22T19:10:01.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T19:10:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:02.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:02 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3307494659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-22T19:10:02.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:02 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3307494659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-22T19:10:02.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:02 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3307494659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-22T19:10:03.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 -- 172.21.15.86:0/2347779688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c100fb0 msgr2=0x7f481c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:03.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 --2- 172.21.15.86:0/2347779688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c100fb0 0x7f481c101390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f480c009990 tx=0x7f480c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 -- 172.21.15.86:0/2347779688 shutdown_connections 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 --2- 172.21.15.86:0/2347779688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481c106090 0x7f481c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 --2- 172.21.15.86:0/2347779688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c100fb0 0x7f481c101390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 --2- 172.21.15.86:0/2347779688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f481c101960 0x7f481c1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 -- 172.21.15.86:0/2347779688 >> 172.21.15.86:0/2347779688 conn(0x7f481c0fc800 msgr2=0x7f481c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 -- 172.21.15.86:0/2347779688 shutdown_connections 2024-03-22T19:10:03.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.384+0000 7f4822c00700 1 -- 172.21.15.86:0/2347779688 wait complete. 2024-03-22T19:10:03.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.385+0000 7f4822c00700 1 Processor -- start 2024-03-22T19:10:03.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 -- start start 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f481c100fb0 0x7f481c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481c101960 0x7f481c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 0x7f481c19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c19e2e0 con 0x7f481c106090 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c19e450 con 0x7f481c100fb0 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4822c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c19e610 con 0x7f481c101960 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4820c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 0x7f481c19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4820c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 0x7f481c19dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50390/0 (socket says 172.21.15.86:50390) 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f481b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481c101960 0x7f481c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f4820c00700 1 -- 172.21.15.86:0/33758617 learned_addr learned my addr 172.21.15.86:0/33758617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.386+0000 7f481be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f481c100fb0 0x7f481c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:03.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4820c00700 1 -- 172.21.15.86:0/33758617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481c101960 msgr2=0x7f481c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4820c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481c101960 0x7f481c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4820c00700 1 -- 172.21.15.86:0/33758617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f481c100fb0 msgr2=0x7f481c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4820c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f481c100fb0 0x7f481c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4820c00700 1 -- 172.21.15.86:0/33758617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0095f0 con 0x7f481c106090 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4820c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 0x7f481c19dcd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f481400ef30 tx=0x7f481400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814018070 con 0x7f481c106090 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c19e890 con 0x7f481c106090 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481400f040 con 0x7f481c106090 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481400cd00 con 0x7f481c106090 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.387+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c1a6150 con 0x7f481c106090 2024-03-22T19:10:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.388+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c102490 con 0x7f481c106090 2024-03-22T19:10:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.390+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f4814003a00 con 0x7f481c106090 2024-03-22T19:10:03.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.392+0000 7f4818c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f48000a0b20 0x7f48000a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:03.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.392+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f48140cbcc0 con 0x7f481c106090 2024-03-22T19:10:03.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.392+0000 7f481be00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f48000a0b20 0x7f48000a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:03.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.393+0000 7f481be00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f48000a0b20 0x7f48000a2fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f480c00b2e0 tx=0x7f480c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:03.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.397+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f48140d2080 con 0x7f481c106090 2024-03-22T19:10:03.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:03 smithi131 ceph-mon[78559]: pgmap v258: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 10 B/s, 1 objects/s recovering 2024-03-22T19:10:03.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:03 smithi086 ceph-mon[38032]: pgmap v258: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 10 B/s, 1 objects/s recovering 2024-03-22T19:10:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:03 smithi167 ceph-mon[47728]: pgmap v258: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 10 B/s, 1 objects/s recovering 2024-03-22T19:10:03.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.596+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f481c19a830 con 0x7f48000a0b20 2024-03-22T19:10:03.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.607+0000 7f4818c00700 1 -- 172.21.15.86:0/33758617 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340945 (secure 0 0 0) 0x7f481c19a830 con 0x7f48000a0b20 2024-03-22T19:10:03.614 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.611+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f48000a0b20 msgr2=0x7f48000a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.611+0000 7f4822c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f48000a0b20 0x7f48000a2fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f480c00b2e0 tx=0x7f480c005960 comp rx=0 tx=0).stop 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.611+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 msgr2=0x7f481c19dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.611+0000 7f4822c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 0x7f481c19dcd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f481400ef30 tx=0x7f481400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 shutdown_connections 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f48000a0b20 0x7f48000a2fe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c106090 0x7f481c19dcd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f481c100fb0 0x7f481c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 --2- 172.21.15.86:0/33758617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481c101960 0x7f481c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 >> 172.21.15.86:0/33758617 conn(0x7f481c0fc800 msgr2=0x7f481c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 shutdown_connections 2024-03-22T19:10:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:03.612+0000 7f4822c00700 1 -- 172.21.15.86:0/33758617 wait complete. 2024-03-22T19:10:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:10:03.999 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-03-22T19:10:02.161032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656571,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":548,"num_read_kb":389,"num_write":412,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335268,"kb_used_data":11000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606556,"statfs":{"total":1151940427776,"available":1151597113344,"internally_reserved":0,"allocated":11264000,"data_stored":7117134,"data_compressed":112425,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.931058"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229812+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:50.229812+0000","last_peered":"2024-03-22T19:09:50.229812+0000","last_clean":"2024-03-22T19:09:50.229812+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:50.229812+0000","last_undegraded":"2024-03-22T19:09:50.229812+0000","last_fullsized":"2024-03-22T19:09:50.229812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225850+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:50.225850+0000","last_peered":"2024-03-22T19:09:50.225850+0000","last_clean":"2024-03-22T19:09:50.225850+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:50.225850+0000","last_undegraded":"2024-03-22T19:09:50.225850+0000","last_fullsized":"2024-03-22T19:09:50.225850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225832+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:50.225832+0000","last_peered":"2024-03-22T19:09:50.225832+0000","last_clean":"2024-03-22T19:09:50.225832+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:50.225832+0000","last_undegraded":"2024-03-22T19:09:50.225832+0000","last_fullsized":"2024-03-22T19:09:50.225832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223923+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:50.223923+0000","last_peered":"2024-03-22T19:09:50.223923+0000","last_clean":"2024-03-22T19:09:50.223923+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:50.223923+0000","last_undegraded":"2024-03-22T19:09:50.223923+0000","last_fullsized":"2024-03-22T19:09:50.223923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231033+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:50.231033+0000","last_peered":"2024-03-22T19:09:50.231033+0000","last_clean":"2024-03-22T19:09:50.231033+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:50.231033+0000","last_undegraded":"2024-03-22T19:09:50.231033+0000","last_fullsized":"2024-03-22T19:09:50.231033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227105+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:50.227105+0000","last_peered":"2024-03-22T19:09:50.227105+0000","last_clean":"2024-03-22T19:09:50.227105+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:50.227105+0000","last_undegraded":"2024-03-22T19:09:50.227105+0000","last_fullsized":"2024-03-22T19:09:50.227105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224692+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:50.224692+0000","last_peered":"2024-03-22T19:09:50.224692+0000","last_clean":"2024-03-22T19:09:50.224692+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:50.224692+0000","last_undegraded":"2024-03-22T19:09:50.224692+0000","last_fullsized":"2024-03-22T19:09:50.224692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230105+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:50.230105+0000","last_peered":"2024-03-22T19:09:50.230105+0000","last_clean":"2024-03-22T19:09:50.230105+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:50.230105+0000","last_undegraded":"2024-03-22T19:09:50.230105+0000","last_fullsized":"2024-03-22T19:09:50.230105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228937+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:50.228937+0000","last_peered":"2024-03-22T19:09:50.228937+0000","last_clean":"2024-03-22T19:09:50.228937+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:50.228937+0000","last_undegraded":"2024-03-22T19:09:50.228937+0000","last_fullsized":"2024-03-22T19:09:50.228937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280115+0000","last_change":"2024-03-22T19:09:51.280115+0000","last_active":"2024-03-22T19:09:51.280115+0000","last_peered":"2024-03-22T19:09:51.280115+0000","last_clean":"2024-03-22T19:09:51.280115+0000","last_became_active":"2024-03-22T19:09:51.279764+0000","last_became_peered":"2024-03-22T19:09:51.279764+0000","last_unstale":"2024-03-22T19:09:51.280115+0000","last_undegraded":"2024-03-22T19:09:51.280115+0000","last_fullsized":"2024-03-22T19:09:51.280115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:51:35.601327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224581+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:50.224581+0000","last_peered":"2024-03-22T19:09:50.224581+0000","last_clean":"2024-03-22T19:09:50.224581+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:50.224581+0000","last_undegraded":"2024-03-22T19:09:50.224581+0000","last_fullsized":"2024-03-22T19:09:50.224581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223728+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:50.223728+0000","last_peered":"2024-03-22T19:09:50.223728+0000","last_clean":"2024-03-22T19:09:50.223728+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:50.223728+0000","last_undegraded":"2024-03-22T19:09:50.223728+0000","last_fullsized":"2024-03-22T19:09:50.223728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221704+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:50.221704+0000","last_peered":"2024-03-22T19:09:50.221704+0000","last_clean":"2024-03-22T19:09:50.221704+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:50.221704+0000","last_undegraded":"2024-03-22T19:09:50.221704+0000","last_fullsized":"2024-03-22T19:09:50.221704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224532+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:50.224532+0000","last_peered":"2024-03-22T19:09:50.224532+0000","last_clean":"2024-03-22T19:09:50.224532+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:50.224532+0000","last_undegraded":"2024-03-22T19:09:50.224532+0000","last_fullsized":"2024-03-22T19:09:50.224532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225345+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:50.225345+0000","last_peered":"2024-03-22T19:09:50.225345+0000","last_clean":"2024-03-22T19:09:50.225345+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:50.225345+0000","last_undegraded":"2024-03-22T19:09:50.225345+0000","last_fullsized":"2024-03-22T19:09:50.225345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228414+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:50.228414+0000","last_peered":"2024-03-22T19:09:50.228414+0000","last_clean":"2024-03-22T19:09:50.228414+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:50.228414+0000","last_undegraded":"2024-03-22T19:09:50.228414+0000","last_fullsized":"2024-03-22T19:09:50.228414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229938+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:50.229938+0000","last_peered":"2024-03-22T19:09:50.229938+0000","last_clean":"2024-03-22T19:09:50.229938+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:50.229938+0000","last_undegraded":"2024-03-22T19:09:50.229938+0000","last_fullsized":"2024-03-22T19:09:50.229938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224855+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:50.224855+0000","last_peered":"2024-03-22T19:09:50.224855+0000","last_clean":"2024-03-22T19:09:50.224855+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:50.224855+0000","last_undegraded":"2024-03-22T19:09:50.224855+0000","last_fullsized":"2024-03-22T19:09:50.224855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226114+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:50.226114+0000","last_peered":"2024-03-22T19:09:50.226114+0000","last_clean":"2024-03-22T19:09:50.226114+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:50.226114+0000","last_undegraded":"2024-03-22T19:09:50.226114+0000","last_fullsized":"2024-03-22T19:09:50.226114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:59.172492+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:09:59.172492+0000","last_peered":"2024-03-22T19:09:59.172492+0000","last_clean":"2024-03-22T19:09:59.172492+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:09:59.172492+0000","last_undegraded":"2024-03-22T19:09:59.172492+0000","last_fullsized":"2024-03-22T19:09:59.172492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230751+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:50.230751+0000","last_peered":"2024-03-22T19:09:50.230751+0000","last_clean":"2024-03-22T19:09:50.230751+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:50.230751+0000","last_undegraded":"2024-03-22T19:09:50.230751+0000","last_fullsized":"2024-03-22T19:09:50.230751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230850+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:50.230850+0000","last_peered":"2024-03-22T19:09:50.230850+0000","last_clean":"2024-03-22T19:09:50.230850+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:50.230850+0000","last_undegraded":"2024-03-22T19:09:50.230850+0000","last_fullsized":"2024-03-22T19:09:50.230850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226153+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:50.226153+0000","last_peered":"2024-03-22T19:09:50.226153+0000","last_clean":"2024-03-22T19:09:50.226153+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:50.226153+0000","last_undegraded":"2024-03-22T19:09:50.226153+0000","last_fullsized":"2024-03-22T19:09:50.226153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231344+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:50.231344+0000","last_peered":"2024-03-22T19:09:50.231344+0000","last_clean":"2024-03-22T19:09:50.231344+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:50.231344+0000","last_undegraded":"2024-03-22T19:09:50.231344+0000","last_fullsized":"2024-03-22T19:09:50.231344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228584+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:50.228584+0000","last_peered":"2024-03-22T19:09:50.228584+0000","last_clean":"2024-03-22T19:09:50.228584+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:50.228584+0000","last_undegraded":"2024-03-22T19:09:50.228584+0000","last_fullsized":"2024-03-22T19:09:50.228584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231109+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:50.231109+0000","last_peered":"2024-03-22T19:09:50.231109+0000","last_clean":"2024-03-22T19:09:50.231109+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:50.231109+0000","last_undegraded":"2024-03-22T19:09:50.231109+0000","last_fullsized":"2024-03-22T19:09:50.231109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232324+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:50.232324+0000","last_peered":"2024-03-22T19:09:50.232324+0000","last_clean":"2024-03-22T19:09:50.232324+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:50.232324+0000","last_undegraded":"2024-03-22T19:09:50.232324+0000","last_fullsized":"2024-03-22T19:09:50.232324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:00.175285+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:10:00.175285+0000","last_peered":"2024-03-22T19:10:00.175285+0000","last_clean":"2024-03-22T19:10:00.175285+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:10:00.175285+0000","last_undegraded":"2024-03-22T19:10:00.175285+0000","last_fullsized":"2024-03-22T19:10:00.175285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232010+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:50.232010+0000","last_peered":"2024-03-22T19:09:50.232010+0000","last_clean":"2024-03-22T19:09:50.232010+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:50.232010+0000","last_undegraded":"2024-03-22T19:09:50.232010+0000","last_fullsized":"2024-03-22T19:09:50.232010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227866+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:50.227866+0000","last_peered":"2024-03-22T19:09:50.227866+0000","last_clean":"2024-03-22T19:09:50.227866+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:50.227866+0000","last_undegraded":"2024-03-22T19:09:50.227866+0000","last_fullsized":"2024-03-22T19:09:50.227866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230805+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:50.230805+0000","last_peered":"2024-03-22T19:09:50.230805+0000","last_clean":"2024-03-22T19:09:50.230805+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:50.230805+0000","last_undegraded":"2024-03-22T19:09:50.230805+0000","last_fullsized":"2024-03-22T19:09:50.230805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221505+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:50.221505+0000","last_peered":"2024-03-22T19:09:50.221505+0000","last_clean":"2024-03-22T19:09:50.221505+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:50.221505+0000","last_undegraded":"2024-03-22T19:09:50.221505+0000","last_fullsized":"2024-03-22T19:09:50.221505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230067+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:50.230067+0000","last_peered":"2024-03-22T19:09:50.230067+0000","last_clean":"2024-03-22T19:09:50.230067+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:50.230067+0000","last_undegraded":"2024-03-22T19:09:50.230067+0000","last_fullsized":"2024-03-22T19:09:50.230067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229198+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:50.229198+0000","last_peered":"2024-03-22T19:09:50.229198+0000","last_clean":"2024-03-22T19:09:50.229198+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:50.229198+0000","last_undegraded":"2024-03-22T19:09:50.229198+0000","last_fullsized":"2024-03-22T19:09:50.229198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224276+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:50.224276+0000","last_peered":"2024-03-22T19:09:50.224276+0000","last_clean":"2024-03-22T19:09:50.224276+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:50.224276+0000","last_undegraded":"2024-03-22T19:09:50.224276+0000","last_fullsized":"2024-03-22T19:09:50.224276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":92,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:57.170609+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:09:57.170609+0000","last_peered":"2024-03-22T19:09:57.170609+0000","last_clean":"2024-03-22T19:09:57.170609+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:09:57.170609+0000","last_undegraded":"2024-03-22T19:09:57.170609+0000","last_fullsized":"2024-03-22T19:09:57.170609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228903+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:50.228903+0000","last_peered":"2024-03-22T19:09:50.228903+0000","last_clean":"2024-03-22T19:09:50.228903+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:50.228903+0000","last_undegraded":"2024-03-22T19:09:50.228903+0000","last_fullsized":"2024-03-22T19:09:50.228903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.438753+0000","last_change":"2024-03-22T19:09:51.438753+0000","last_active":"2024-03-22T19:09:51.438753+0000","last_peered":"2024-03-22T19:09:51.438753+0000","last_clean":"2024-03-22T19:09:51.438753+0000","last_became_active":"2024-03-22T19:09:51.413301+0000","last_became_peered":"2024-03-22T19:09:51.413301+0000","last_unstale":"2024-03-22T19:09:51.438753+0000","last_undegraded":"2024-03-22T19:09:51.438753+0000","last_fullsized":"2024-03-22T19:09:51.438753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:49.994282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226106+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:50.226106+0000","last_peered":"2024-03-22T19:09:50.226106+0000","last_clean":"2024-03-22T19:09:50.226106+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:50.226106+0000","last_undegraded":"2024-03-22T19:09:50.226106+0000","last_fullsized":"2024-03-22T19:09:50.226106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232420+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:50.232420+0000","last_peered":"2024-03-22T19:09:50.232420+0000","last_clean":"2024-03-22T19:09:50.232420+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:50.232420+0000","last_undegraded":"2024-03-22T19:09:50.232420+0000","last_fullsized":"2024-03-22T19:09:50.232420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228700+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:50.228700+0000","last_peered":"2024-03-22T19:09:50.228700+0000","last_clean":"2024-03-22T19:09:50.228700+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:50.228700+0000","last_undegraded":"2024-03-22T19:09:50.228700+0000","last_fullsized":"2024-03-22T19:09:50.228700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'29","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:57.182146+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:09:57.182146+0000","last_peered":"2024-03-22T19:09:57.182146+0000","last_clean":"2024-03-22T19:09:57.182146+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:09:57.182146+0000","last_undegraded":"2024-03-22T19:09:57.182146+0000","last_fullsized":"2024-03-22T19:09:57.182146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":98,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230888+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:50.230888+0000","last_peered":"2024-03-22T19:09:50.230888+0000","last_clean":"2024-03-22T19:09:50.230888+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:50.230888+0000","last_undegraded":"2024-03-22T19:09:50.230888+0000","last_fullsized":"2024-03-22T19:09:50.230888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226121+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:50.226121+0000","last_peered":"2024-03-22T19:09:50.226121+0000","last_clean":"2024-03-22T19:09:50.226121+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:50.226121+0000","last_undegraded":"2024-03-22T19:09:50.226121+0000","last_fullsized":"2024-03-22T19:09:50.226121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221747+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:50.221747+0000","last_peered":"2024-03-22T19:09:50.221747+0000","last_clean":"2024-03-22T19:09:50.221747+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:50.221747+0000","last_undegraded":"2024-03-22T19:09:50.221747+0000","last_fullsized":"2024-03-22T19:09:50.221747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230590+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:50.230590+0000","last_peered":"2024-03-22T19:09:50.230590+0000","last_clean":"2024-03-22T19:09:50.230590+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:50.230590+0000","last_undegraded":"2024-03-22T19:09:50.230590+0000","last_fullsized":"2024-03-22T19:09:50.230590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224672+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:50.224672+0000","last_peered":"2024-03-22T19:09:50.224672+0000","last_clean":"2024-03-22T19:09:50.224672+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:50.224672+0000","last_undegraded":"2024-03-22T19:09:50.224672+0000","last_fullsized":"2024-03-22T19:09:50.224672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:58.172398+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:09:58.172398+0000","last_peered":"2024-03-22T19:09:58.172398+0000","last_clean":"2024-03-22T19:09:58.172398+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:09:58.172398+0000","last_undegraded":"2024-03-22T19:09:58.172398+0000","last_fullsized":"2024-03-22T19:09:58.172398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.435708+0000","last_change":"2024-03-22T19:09:51.435708+0000","last_active":"2024-03-22T19:09:51.435708+0000","last_peered":"2024-03-22T19:09:51.435708+0000","last_clean":"2024-03-22T19:09:51.435708+0000","last_became_active":"2024-03-22T19:09:51.412550+0000","last_became_peered":"2024-03-22T19:09:51.412550+0000","last_unstale":"2024-03-22T19:09:51.435708+0000","last_undegraded":"2024-03-22T19:09:51.435708+0000","last_fullsized":"2024-03-22T19:09:51.435708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:11:44.092847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229832+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:50.229832+0000","last_peered":"2024-03-22T19:09:50.229832+0000","last_clean":"2024-03-22T19:09:50.229832+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:50.229832+0000","last_undegraded":"2024-03-22T19:09:50.229832+0000","last_fullsized":"2024-03-22T19:09:50.229832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231493+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:50.231493+0000","last_peered":"2024-03-22T19:09:50.231493+0000","last_clean":"2024-03-22T19:09:50.231493+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:50.231493+0000","last_undegraded":"2024-03-22T19:09:50.231493+0000","last_fullsized":"2024-03-22T19:09:50.231493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229589+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:50.229589+0000","last_peered":"2024-03-22T19:09:50.229589+0000","last_clean":"2024-03-22T19:09:50.229589+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:50.229589+0000","last_undegraded":"2024-03-22T19:09:50.229589+0000","last_fullsized":"2024-03-22T19:09:50.229589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225716+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:50.225716+0000","last_peered":"2024-03-22T19:09:50.225716+0000","last_clean":"2024-03-22T19:09:50.225716+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:50.225716+0000","last_undegraded":"2024-03-22T19:09:50.225716+0000","last_fullsized":"2024-03-22T19:09:50.225716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227355+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:50.227355+0000","last_peered":"2024-03-22T19:09:50.227355+0000","last_clean":"2024-03-22T19:09:50.227355+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:50.227355+0000","last_undegraded":"2024-03-22T19:09:50.227355+0000","last_fullsized":"2024-03-22T19:09:50.227355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:00.174580+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:10:00.174580+0000","last_peered":"2024-03-22T19:10:00.174580+0000","last_clean":"2024-03-22T19:10:00.174580+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:10:00.174580+0000","last_undegraded":"2024-03-22T19:10:00.174580+0000","last_fullsized":"2024-03-22T19:10:00.174580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227464+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:50.227464+0000","last_peered":"2024-03-22T19:09:50.227464+0000","last_clean":"2024-03-22T19:09:50.227464+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:50.227464+0000","last_undegraded":"2024-03-22T19:09:50.227464+0000","last_fullsized":"2024-03-22T19:09:50.227464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229382+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:50.229382+0000","last_peered":"2024-03-22T19:09:50.229382+0000","last_clean":"2024-03-22T19:09:50.229382+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:50.229382+0000","last_undegraded":"2024-03-22T19:09:50.229382+0000","last_fullsized":"2024-03-22T19:09:50.229382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:58.171782+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:09:58.171782+0000","last_peered":"2024-03-22T19:09:58.171782+0000","last_clean":"2024-03-22T19:09:58.171782+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:09:58.171782+0000","last_undegraded":"2024-03-22T19:09:58.171782+0000","last_fullsized":"2024-03-22T19:09:58.171782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230745+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:50.230745+0000","last_peered":"2024-03-22T19:09:50.230745+0000","last_clean":"2024-03-22T19:09:50.230745+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:50.230745+0000","last_undegraded":"2024-03-22T19:09:50.230745+0000","last_fullsized":"2024-03-22T19:09:50.230745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223407+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:50.223407+0000","last_peered":"2024-03-22T19:09:50.223407+0000","last_clean":"2024-03-22T19:09:50.223407+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:50.223407+0000","last_undegraded":"2024-03-22T19:09:50.223407+0000","last_fullsized":"2024-03-22T19:09:50.223407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224404+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:50.224404+0000","last_peered":"2024-03-22T19:09:50.224404+0000","last_clean":"2024-03-22T19:09:50.224404+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:50.224404+0000","last_undegraded":"2024-03-22T19:09:50.224404+0000","last_fullsized":"2024-03-22T19:09:50.224404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221409+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:50.221409+0000","last_peered":"2024-03-22T19:09:50.221409+0000","last_clean":"2024-03-22T19:09:50.221409+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:50.221409+0000","last_undegraded":"2024-03-22T19:09:50.221409+0000","last_fullsized":"2024-03-22T19:09:50.221409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224573+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:50.224573+0000","last_peered":"2024-03-22T19:09:50.224573+0000","last_clean":"2024-03-22T19:09:50.224573+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:50.224573+0000","last_undegraded":"2024-03-22T19:09:50.224573+0000","last_fullsized":"2024-03-22T19:09:50.224573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231242+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:50.231242+0000","last_peered":"2024-03-22T19:09:50.231242+0000","last_clean":"2024-03-22T19:09:50.231242+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:50.231242+0000","last_undegraded":"2024-03-22T19:09:50.231242+0000","last_fullsized":"2024-03-22T19:09:50.231242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221846+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:50.221846+0000","last_peered":"2024-03-22T19:09:50.221846+0000","last_clean":"2024-03-22T19:09:50.221846+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:50.221846+0000","last_undegraded":"2024-03-22T19:09:50.221846+0000","last_fullsized":"2024-03-22T19:09:50.221846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224775+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:50.224775+0000","last_peered":"2024-03-22T19:09:50.224775+0000","last_clean":"2024-03-22T19:09:50.224775+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:50.224775+0000","last_undegraded":"2024-03-22T19:09:50.224775+0000","last_fullsized":"2024-03-22T19:09:50.224775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230722+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:50.230722+0000","last_peered":"2024-03-22T19:09:50.230722+0000","last_clean":"2024-03-22T19:09:50.230722+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:50.230722+0000","last_undegraded":"2024-03-22T19:09:50.230722+0000","last_fullsized":"2024-03-22T19:09:50.230722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223816+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:50.223816+0000","last_peered":"2024-03-22T19:09:50.223816+0000","last_clean":"2024-03-22T19:09:50.223816+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:50.223816+0000","last_undegraded":"2024-03-22T19:09:50.223816+0000","last_fullsized":"2024-03-22T19:09:50.223816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225947+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:50.225947+0000","last_peered":"2024-03-22T19:09:50.225947+0000","last_clean":"2024-03-22T19:09:50.225947+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:50.225947+0000","last_undegraded":"2024-03-22T19:09:50.225947+0000","last_fullsized":"2024-03-22T19:09:50.225947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228471+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:50.228471+0000","last_peered":"2024-03-22T19:09:50.228471+0000","last_clean":"2024-03-22T19:09:50.228471+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:50.228471+0000","last_undegraded":"2024-03-22T19:09:50.228471+0000","last_fullsized":"2024-03-22T19:09:50.228471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230778+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:50.230778+0000","last_peered":"2024-03-22T19:09:50.230778+0000","last_clean":"2024-03-22T19:09:50.230778+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:50.230778+0000","last_undegraded":"2024-03-22T19:09:50.230778+0000","last_fullsized":"2024-03-22T19:09:50.230778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230201+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:50.230201+0000","last_peered":"2024-03-22T19:09:50.230201+0000","last_clean":"2024-03-22T19:09:50.230201+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:50.230201+0000","last_undegraded":"2024-03-22T19:09:50.230201+0000","last_fullsized":"2024-03-22T19:09:50.230201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229252+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:50.229252+0000","last_peered":"2024-03-22T19:09:50.229252+0000","last_clean":"2024-03-22T19:09:50.229252+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:50.229252+0000","last_undegraded":"2024-03-22T19:09:50.229252+0000","last_fullsized":"2024-03-22T19:09:50.229252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.241586+0000","last_change":"2024-03-22T19:09:51.241586+0000","last_active":"2024-03-22T19:09:51.241586+0000","last_peered":"2024-03-22T19:09:51.241586+0000","last_clean":"2024-03-22T19:09:51.241586+0000","last_became_active":"2024-03-22T19:09:51.235327+0000","last_became_peered":"2024-03-22T19:09:51.235327+0000","last_unstale":"2024-03-22T19:09:51.241586+0000","last_undegraded":"2024-03-22T19:09:51.241586+0000","last_fullsized":"2024-03-22T19:09:51.241586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:43:59.307732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230257+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:50.230257+0000","last_peered":"2024-03-22T19:09:50.230257+0000","last_clean":"2024-03-22T19:09:50.230257+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:50.230257+0000","last_undegraded":"2024-03-22T19:09:50.230257+0000","last_fullsized":"2024-03-22T19:09:50.230257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231141+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:50.231141+0000","last_peered":"2024-03-22T19:09:50.231141+0000","last_clean":"2024-03-22T19:09:50.231141+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:50.231141+0000","last_undegraded":"2024-03-22T19:09:50.231141+0000","last_fullsized":"2024-03-22T19:09:50.231141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224983+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:50.224983+0000","last_peered":"2024-03-22T19:09:50.224983+0000","last_clean":"2024-03-22T19:09:50.224983+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:50.224983+0000","last_undegraded":"2024-03-22T19:09:50.224983+0000","last_fullsized":"2024-03-22T19:09:50.224983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:59.173923+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:09:59.173923+0000","last_peered":"2024-03-22T19:09:59.173923+0000","last_clean":"2024-03-22T19:09:59.173923+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:59.173923+0000","last_undegraded":"2024-03-22T19:09:59.173923+0000","last_fullsized":"2024-03-22T19:09:59.173923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224702+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:50.224702+0000","last_peered":"2024-03-22T19:09:50.224702+0000","last_clean":"2024-03-22T19:09:50.224702+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:50.224702+0000","last_undegraded":"2024-03-22T19:09:50.224702+0000","last_fullsized":"2024-03-22T19:09:50.224702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232307+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:50.232307+0000","last_peered":"2024-03-22T19:09:50.232307+0000","last_clean":"2024-03-22T19:09:50.232307+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:50.232307+0000","last_undegraded":"2024-03-22T19:09:50.232307+0000","last_fullsized":"2024-03-22T19:09:50.232307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221786+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:50.221786+0000","last_peered":"2024-03-22T19:09:50.221786+0000","last_clean":"2024-03-22T19:09:50.221786+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:50.221786+0000","last_undegraded":"2024-03-22T19:09:50.221786+0000","last_fullsized":"2024-03-22T19:09:50.221786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224337+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:50.224337+0000","last_peered":"2024-03-22T19:09:50.224337+0000","last_clean":"2024-03-22T19:09:50.224337+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:50.224337+0000","last_undegraded":"2024-03-22T19:09:50.224337+0000","last_fullsized":"2024-03-22T19:09:50.224337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229345+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:50.229345+0000","last_peered":"2024-03-22T19:09:50.229345+0000","last_clean":"2024-03-22T19:09:50.229345+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:50.229345+0000","last_undegraded":"2024-03-22T19:09:50.229345+0000","last_fullsized":"2024-03-22T19:09:50.229345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.412328+0000","last_change":"2024-03-22T19:09:51.412328+0000","last_active":"2024-03-22T19:09:51.412328+0000","last_peered":"2024-03-22T19:09:51.412328+0000","last_clean":"2024-03-22T19:09:51.412328+0000","last_became_active":"2024-03-22T19:09:51.411942+0000","last_became_peered":"2024-03-22T19:09:51.411942+0000","last_unstale":"2024-03-22T19:09:51.412328+0000","last_undegraded":"2024-03-22T19:09:51.412328+0000","last_fullsized":"2024-03-22T19:09:51.412328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:39:48.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225949+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:50.225949+0000","last_peered":"2024-03-22T19:09:50.225949+0000","last_clean":"2024-03-22T19:09:50.225949+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:50.225949+0000","last_undegraded":"2024-03-22T19:09:50.225949+0000","last_fullsized":"2024-03-22T19:09:50.225949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225646+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:50.225646+0000","last_peered":"2024-03-22T19:09:50.225646+0000","last_clean":"2024-03-22T19:09:50.225646+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:50.225646+0000","last_undegraded":"2024-03-22T19:09:50.225646+0000","last_fullsized":"2024-03-22T19:09:50.225646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280034+0000","last_change":"2024-03-22T19:09:51.280034+0000","last_active":"2024-03-22T19:09:51.280034+0000","last_peered":"2024-03-22T19:09:51.280034+0000","last_clean":"2024-03-22T19:09:51.280034+0000","last_became_active":"2024-03-22T19:09:51.279604+0000","last_became_peered":"2024-03-22T19:09:51.279604+0000","last_unstale":"2024-03-22T19:09:51.280034+0000","last_undegraded":"2024-03-22T19:09:51.280034+0000","last_fullsized":"2024-03-22T19:09:51.280034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:38:59.518086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229596+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:50.229596+0000","last_peered":"2024-03-22T19:09:50.229596+0000","last_clean":"2024-03-22T19:09:50.229596+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:50.229596+0000","last_undegraded":"2024-03-22T19:09:50.229596+0000","last_fullsized":"2024-03-22T19:09:50.229596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226600+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:50.226600+0000","last_peered":"2024-03-22T19:09:50.226600+0000","last_clean":"2024-03-22T19:09:50.226600+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:50.226600+0000","last_undegraded":"2024-03-22T19:09:50.226600+0000","last_fullsized":"2024-03-22T19:09:50.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230693+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:50.230693+0000","last_peered":"2024-03-22T19:09:50.230693+0000","last_clean":"2024-03-22T19:09:50.230693+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:50.230693+0000","last_undegraded":"2024-03-22T19:09:50.230693+0000","last_fullsized":"2024-03-22T19:09:50.230693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229186+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:50.229186+0000","last_peered":"2024-03-22T19:09:50.229186+0000","last_clean":"2024-03-22T19:09:50.229186+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:50.229186+0000","last_undegraded":"2024-03-22T19:09:50.229186+0000","last_fullsized":"2024-03-22T19:09:50.229186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230140+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:50.230140+0000","last_peered":"2024-03-22T19:09:50.230140+0000","last_clean":"2024-03-22T19:09:50.230140+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:50.230140+0000","last_undegraded":"2024-03-22T19:09:50.230140+0000","last_fullsized":"2024-03-22T19:09:50.230140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228785+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:50.228785+0000","last_peered":"2024-03-22T19:09:50.228785+0000","last_clean":"2024-03-22T19:09:50.228785+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:50.228785+0000","last_undegraded":"2024-03-22T19:09:50.228785+0000","last_fullsized":"2024-03-22T19:09:50.228785+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221650+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:50.221650+0000","last_peered":"2024-03-22T19:09:50.221650+0000","last_clean":"2024-03-22T19:09:50.221650+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:50.221650+0000","last_undegraded":"2024-03-22T19:09:50.221650+0000","last_fullsized":"2024-03-22T19:09:50.221650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229710+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:50.229710+0000","last_peered":"2024-03-22T19:09:50.229710+0000","last_clean":"2024-03-22T19:09:50.229710+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:50.229710+0000","last_undegraded":"2024-03-22T19:09:50.229710+0000","last_fullsized":"2024-03-22T19:09:50.229710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65969,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":422,"num_read_kb":280,"num_write":229,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":197907,"data_compressed":5466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":430423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.627},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77000000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.82699999999999996}]}]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080356,"data_compressed":37475,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.433}]},{"osd":2,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.626},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.498},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.875},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68400000000000005}]},{"osd":9,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.80600000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:09:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.753}]}]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":430289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.30599999999999999}]},{"osd":1,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.497}]},{"osd":2,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.625},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.44}]},{"osd":7,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.66100000000000003}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.68799999999999994}]}]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.377}]},{"osd":3,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.875,"15min":0.875},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.63500000000000001},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.622}]},{"osd":7,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66300000000000003}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1022761,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41799999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.60799999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.25700000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86699999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82899999999999996,"5min":1.111,"15min":1.111},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78600000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82399999999999995,"5min":0.878,"15min":0.878},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96399999999999997,"5min":1.155,"15min":1.155},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68200000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.878},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.80400000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83599999999999997}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77800000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72099999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.309}]},{"osd":1,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81699999999999995,"5min":0.875,"15min":0.875},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.369,"15min":1.369},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.76200000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.53400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71399999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78900000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.749,"15min":0.76100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84599999999999997,"5min":1.55,"15min":1.55},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85799999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Mar 22 19:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.502},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.75},"min":{"1min":0.53200000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:09:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.82299999999999995,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.68600000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.59499999999999997}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":489909,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66600000000000004,"15min":0.77700000000000002},"min":{"1min":0.42999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77700000000000002,"5min":6.71,"15min":6.71},"last":0.502}]},{"osd":2,"last update":"Fri Mar 22 19:09:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Mar 22 19:09:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:09:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.308,"5min":0.254,"15min":0.254},"max":{"1min":0.59699999999999998,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.443},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.45500000000000002,"15min":0.434},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80700000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.38900000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.874,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.623}]},{"osd":7,"last update":"Fri Mar 22 19:09:27 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46600000000000003}]},{"osd":8,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.63,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82399999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65900000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75}]},{"osd":10,"last update":"Fri Mar 22 19:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":431297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:09:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:08:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45800000000000002,"15min":0.44400000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.748}]},{"osd":4,"last update":"Fri Mar 22 19:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64400000000000002,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96099999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":3.044},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":3.0590000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85599999999999998}]},{"osd":11,"last update":"Fri Mar 22 19:09:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":3.0790000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":489670,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.50600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:08:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.51500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.437},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65300000000000002,"5min":1.448,"15min":1.448},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:09:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63500000000000001}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59999999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.625},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78000000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.42699999999999999},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.024,"5min":1.244,"15min":1.244},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.367,"15min":1.367},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56000000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84299999999999997,"5min":1.208,"15min":1.208},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.417,"15min":1.417},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97599999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Mar 22 19:09:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.077,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94299999999999995,"5min":1.641,"15min":1.641},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1220000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.749,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.74299999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953518,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.374,"15min":0.36199999999999999},"min":{"1min":0.29199999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38900000000000001,"15min":0.38},"min":{"1min":0.30399999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34599999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.626}]},{"osd":3,"last update":"Fri Mar 22 19:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82299999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83599999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.65600000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.495}]},{"osd":7,"last update":"Fri Mar 22 19:09:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.38,"5min":0.313,"15min":0.313},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.436,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92900000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.755}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.73599999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.65700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1022485,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67600000000000005,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85799999999999998,"5min":3.887,"15min":3.887},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.66200000000000003,"15min":0.71299999999999997},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.376}]},{"osd":1,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47499999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38,"15min":0.37},"min":{"1min":0.32800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.371},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.29599999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.433}]},{"osd":6,"last update":"Fri Mar 22 19:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.76300000000000001,"15min":0.78700000000000003},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83999999999999997,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 22 19:09:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59599999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:09:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61051,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61503,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61035,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:10:04.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T19:10:04.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:04 smithi086 ceph-mon[38032]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:04.368 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:04.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:04 smithi131 ceph-mon[78559]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:04.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:04 smithi167 ceph-mon[47728]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:05.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:05 smithi086 ceph-mon[38032]: pgmap v259: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-22T19:10:05.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:05 smithi131 ceph-mon[78559]: pgmap v259: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-22T19:10:05.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:05 smithi167 ceph-mon[47728]: pgmap v259: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-22T19:10:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 -- 172.21.15.86:0/1719513217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 msgr2=0x7ff1041057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 --2- 172.21.15.86:0/1719513217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff1041057e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009990 tx=0x7ff0f0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 -- 172.21.15.86:0/1719513217 shutdown_connections 2024-03-22T19:10:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 --2- 172.21.15.86:0/1719513217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff104105ec0 0x7ff10410da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 --2- 172.21.15.86:0/1719513217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104100de0 0x7ff1041011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 --2- 172.21.15.86:0/1719513217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff1041057e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 -- 172.21.15.86:0/1719513217 >> 172.21.15.86:0/1719513217 conn(0x7ff1040fc6b0 msgr2=0x7ff1040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.592+0000 7ff10ae00700 1 -- 172.21.15.86:0/1719513217 shutdown_connections 2024-03-22T19:10:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.593+0000 7ff10ae00700 1 -- 172.21.15.86:0/1719513217 wait complete. 2024-03-22T19:10:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.593+0000 7ff10ae00700 1 Processor -- start 2024-03-22T19:10:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 -- start start 2024-03-22T19:10:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff104100de0 0x7ff104199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff104199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104105ec0 0x7ff10419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10419e140 con 0x7ff104101790 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10419e2b0 con 0x7ff104100de0 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10ae00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10419e470 con 0x7ff104105ec0 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff109000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff104199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff109000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff104199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50410/0 (socket says 172.21.15.86:50410) 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff109000700 1 -- 172.21.15.86:0/2181463831 learned_addr learned my addr 172.21.15.86:0/2181463831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff109a00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff104100de0 0x7ff104199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.594+0000 7ff10a400700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104105ec0 0x7ff10419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff109000700 1 -- 172.21.15.86:0/2181463831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104105ec0 msgr2=0x7ff10419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff109000700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104105ec0 0x7ff10419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff109000700 1 -- 172.21.15.86:0/2181463831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff104100de0 msgr2=0x7ff104199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff109000700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff104100de0 0x7ff104199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff109000700 1 -- 172.21.15.86:0/2181463831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f00095f0 con 0x7ff104101790 2024-03-22T19:10:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff109000700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff104199780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00095c0 tx=0x7ff0f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f001d070 con 0x7ff104101790 2024-03-22T19:10:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f0005050 con 0x7ff104101790 2024-03-22T19:10:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10419e6f0 con 0x7ff104101790 2024-03-22T19:10:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0004a70 con 0x7ff104101790 2024-03-22T19:10:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.595+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1041a5fb0 con 0x7ff104101790 2024-03-22T19:10:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.596+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1041022c0 con 0x7ff104101790 2024-03-22T19:10:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.598+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7ff0f00051c0 con 0x7ff104101790 2024-03-22T19:10:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.599+0000 7ff102000700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff0ec0a0ba0 0x7ff0ec0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.600+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff0f00ccec0 con 0x7ff104101790 2024-03-22T19:10:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.600+0000 7ff109a00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff0ec0a0ba0 0x7ff0ec0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.600+0000 7ff109a00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff0ec0a0ba0 0x7ff0ec0a3060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8005950 tx=0x7ff0f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.603+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff0f00967b0 con 0x7ff104101790 2024-03-22T19:10:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.801+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff10419a550 con 0x7ff0ec0a0ba0 2024-03-22T19:10:05.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.811+0000 7ff102000700 1 -- 172.21.15.86:0/2181463831 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340891 (secure 0 0 0) 0x7ff10419a550 con 0x7ff0ec0a0ba0 2024-03-22T19:10:05.818 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff0ec0a0ba0 msgr2=0x7ff0ec0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff0ec0a0ba0 0x7ff0ec0a3060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8005950 tx=0x7ff0f800b410 comp rx=0 tx=0).stop 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 msgr2=0x7ff104199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff104199780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00095c0 tx=0x7ff0f000f690 comp rx=0 tx=0).stop 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 shutdown_connections 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff0ec0a0ba0 0x7ff0ec0a3060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104105ec0 0x7ff10419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff104100de0 0x7ff104199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 --2- 172.21.15.86:0/2181463831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff104101790 0x7ff104199780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 >> 172.21.15.86:0/2181463831 conn(0x7ff1040fc6b0 msgr2=0x7ff1040fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 shutdown_connections 2024-03-22T19:10:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:05.816+0000 7ff10ae00700 1 -- 172.21.15.86:0/2181463831 wait complete. 2024-03-22T19:10:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:10:06.221 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-03-22T19:10:04.161856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656571,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":568,"num_read_kb":404,"num_write":412,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335268,"kb_used_data":11000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606556,"statfs":{"total":1151940427776,"available":1151597113344,"internally_reserved":0,"allocated":11264000,"data_stored":7117134,"data_compressed":112425,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003212"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229812+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:50.229812+0000","last_peered":"2024-03-22T19:09:50.229812+0000","last_clean":"2024-03-22T19:09:50.229812+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:50.229812+0000","last_undegraded":"2024-03-22T19:09:50.229812+0000","last_fullsized":"2024-03-22T19:09:50.229812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225850+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:50.225850+0000","last_peered":"2024-03-22T19:09:50.225850+0000","last_clean":"2024-03-22T19:09:50.225850+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:50.225850+0000","last_undegraded":"2024-03-22T19:09:50.225850+0000","last_fullsized":"2024-03-22T19:09:50.225850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225832+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:50.225832+0000","last_peered":"2024-03-22T19:09:50.225832+0000","last_clean":"2024-03-22T19:09:50.225832+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:50.225832+0000","last_undegraded":"2024-03-22T19:09:50.225832+0000","last_fullsized":"2024-03-22T19:09:50.225832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223923+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:50.223923+0000","last_peered":"2024-03-22T19:09:50.223923+0000","last_clean":"2024-03-22T19:09:50.223923+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:50.223923+0000","last_undegraded":"2024-03-22T19:09:50.223923+0000","last_fullsized":"2024-03-22T19:09:50.223923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231033+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:50.231033+0000","last_peered":"2024-03-22T19:09:50.231033+0000","last_clean":"2024-03-22T19:09:50.231033+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:50.231033+0000","last_undegraded":"2024-03-22T19:09:50.231033+0000","last_fullsized":"2024-03-22T19:09:50.231033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227105+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:50.227105+0000","last_peered":"2024-03-22T19:09:50.227105+0000","last_clean":"2024-03-22T19:09:50.227105+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:50.227105+0000","last_undegraded":"2024-03-22T19:09:50.227105+0000","last_fullsized":"2024-03-22T19:09:50.227105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224692+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:50.224692+0000","last_peered":"2024-03-22T19:09:50.224692+0000","last_clean":"2024-03-22T19:09:50.224692+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:50.224692+0000","last_undegraded":"2024-03-22T19:09:50.224692+0000","last_fullsized":"2024-03-22T19:09:50.224692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230105+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:50.230105+0000","last_peered":"2024-03-22T19:09:50.230105+0000","last_clean":"2024-03-22T19:09:50.230105+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:50.230105+0000","last_undegraded":"2024-03-22T19:09:50.230105+0000","last_fullsized":"2024-03-22T19:09:50.230105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228937+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:50.228937+0000","last_peered":"2024-03-22T19:09:50.228937+0000","last_clean":"2024-03-22T19:09:50.228937+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:50.228937+0000","last_undegraded":"2024-03-22T19:09:50.228937+0000","last_fullsized":"2024-03-22T19:09:50.228937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280115+0000","last_change":"2024-03-22T19:09:51.280115+0000","last_active":"2024-03-22T19:09:51.280115+0000","last_peered":"2024-03-22T19:09:51.280115+0000","last_clean":"2024-03-22T19:09:51.280115+0000","last_became_active":"2024-03-22T19:09:51.279764+0000","last_became_peered":"2024-03-22T19:09:51.279764+0000","last_unstale":"2024-03-22T19:09:51.280115+0000","last_undegraded":"2024-03-22T19:09:51.280115+0000","last_fullsized":"2024-03-22T19:09:51.280115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:51:35.601327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224581+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:50.224581+0000","last_peered":"2024-03-22T19:09:50.224581+0000","last_clean":"2024-03-22T19:09:50.224581+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:50.224581+0000","last_undegraded":"2024-03-22T19:09:50.224581+0000","last_fullsized":"2024-03-22T19:09:50.224581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223728+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:50.223728+0000","last_peered":"2024-03-22T19:09:50.223728+0000","last_clean":"2024-03-22T19:09:50.223728+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:50.223728+0000","last_undegraded":"2024-03-22T19:09:50.223728+0000","last_fullsized":"2024-03-22T19:09:50.223728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221704+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:50.221704+0000","last_peered":"2024-03-22T19:09:50.221704+0000","last_clean":"2024-03-22T19:09:50.221704+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:50.221704+0000","last_undegraded":"2024-03-22T19:09:50.221704+0000","last_fullsized":"2024-03-22T19:09:50.221704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224532+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:50.224532+0000","last_peered":"2024-03-22T19:09:50.224532+0000","last_clean":"2024-03-22T19:09:50.224532+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:50.224532+0000","last_undegraded":"2024-03-22T19:09:50.224532+0000","last_fullsized":"2024-03-22T19:09:50.224532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225345+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:50.225345+0000","last_peered":"2024-03-22T19:09:50.225345+0000","last_clean":"2024-03-22T19:09:50.225345+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:50.225345+0000","last_undegraded":"2024-03-22T19:09:50.225345+0000","last_fullsized":"2024-03-22T19:09:50.225345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228414+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:50.228414+0000","last_peered":"2024-03-22T19:09:50.228414+0000","last_clean":"2024-03-22T19:09:50.228414+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:50.228414+0000","last_undegraded":"2024-03-22T19:09:50.228414+0000","last_fullsized":"2024-03-22T19:09:50.228414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229938+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:50.229938+0000","last_peered":"2024-03-22T19:09:50.229938+0000","last_clean":"2024-03-22T19:09:50.229938+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:50.229938+0000","last_undegraded":"2024-03-22T19:09:50.229938+0000","last_fullsized":"2024-03-22T19:09:50.229938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224855+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:50.224855+0000","last_peered":"2024-03-22T19:09:50.224855+0000","last_clean":"2024-03-22T19:09:50.224855+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:50.224855+0000","last_undegraded":"2024-03-22T19:09:50.224855+0000","last_fullsized":"2024-03-22T19:09:50.224855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226114+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:50.226114+0000","last_peered":"2024-03-22T19:09:50.226114+0000","last_clean":"2024-03-22T19:09:50.226114+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:50.226114+0000","last_undegraded":"2024-03-22T19:09:50.226114+0000","last_fullsized":"2024-03-22T19:09:50.226114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:59.172492+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:09:59.172492+0000","last_peered":"2024-03-22T19:09:59.172492+0000","last_clean":"2024-03-22T19:09:59.172492+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:09:59.172492+0000","last_undegraded":"2024-03-22T19:09:59.172492+0000","last_fullsized":"2024-03-22T19:09:59.172492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230751+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:50.230751+0000","last_peered":"2024-03-22T19:09:50.230751+0000","last_clean":"2024-03-22T19:09:50.230751+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:50.230751+0000","last_undegraded":"2024-03-22T19:09:50.230751+0000","last_fullsized":"2024-03-22T19:09:50.230751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230850+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:50.230850+0000","last_peered":"2024-03-22T19:09:50.230850+0000","last_clean":"2024-03-22T19:09:50.230850+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:50.230850+0000","last_undegraded":"2024-03-22T19:09:50.230850+0000","last_fullsized":"2024-03-22T19:09:50.230850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226153+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:50.226153+0000","last_peered":"2024-03-22T19:09:50.226153+0000","last_clean":"2024-03-22T19:09:50.226153+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:50.226153+0000","last_undegraded":"2024-03-22T19:09:50.226153+0000","last_fullsized":"2024-03-22T19:09:50.226153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231344+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:50.231344+0000","last_peered":"2024-03-22T19:09:50.231344+0000","last_clean":"2024-03-22T19:09:50.231344+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:50.231344+0000","last_undegraded":"2024-03-22T19:09:50.231344+0000","last_fullsized":"2024-03-22T19:09:50.231344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228584+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:50.228584+0000","last_peered":"2024-03-22T19:09:50.228584+0000","last_clean":"2024-03-22T19:09:50.228584+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:50.228584+0000","last_undegraded":"2024-03-22T19:09:50.228584+0000","last_fullsized":"2024-03-22T19:09:50.228584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231109+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:50.231109+0000","last_peered":"2024-03-22T19:09:50.231109+0000","last_clean":"2024-03-22T19:09:50.231109+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:50.231109+0000","last_undegraded":"2024-03-22T19:09:50.231109+0000","last_fullsized":"2024-03-22T19:09:50.231109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232324+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:50.232324+0000","last_peered":"2024-03-22T19:09:50.232324+0000","last_clean":"2024-03-22T19:09:50.232324+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:50.232324+0000","last_undegraded":"2024-03-22T19:09:50.232324+0000","last_fullsized":"2024-03-22T19:09:50.232324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:00.175285+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:10:00.175285+0000","last_peered":"2024-03-22T19:10:00.175285+0000","last_clean":"2024-03-22T19:10:00.175285+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:10:00.175285+0000","last_undegraded":"2024-03-22T19:10:00.175285+0000","last_fullsized":"2024-03-22T19:10:00.175285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232010+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:50.232010+0000","last_peered":"2024-03-22T19:09:50.232010+0000","last_clean":"2024-03-22T19:09:50.232010+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:50.232010+0000","last_undegraded":"2024-03-22T19:09:50.232010+0000","last_fullsized":"2024-03-22T19:09:50.232010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227866+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:50.227866+0000","last_peered":"2024-03-22T19:09:50.227866+0000","last_clean":"2024-03-22T19:09:50.227866+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:50.227866+0000","last_undegraded":"2024-03-22T19:09:50.227866+0000","last_fullsized":"2024-03-22T19:09:50.227866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230805+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:50.230805+0000","last_peered":"2024-03-22T19:09:50.230805+0000","last_clean":"2024-03-22T19:09:50.230805+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:50.230805+0000","last_undegraded":"2024-03-22T19:09:50.230805+0000","last_fullsized":"2024-03-22T19:09:50.230805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221505+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:50.221505+0000","last_peered":"2024-03-22T19:09:50.221505+0000","last_clean":"2024-03-22T19:09:50.221505+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:50.221505+0000","last_undegraded":"2024-03-22T19:09:50.221505+0000","last_fullsized":"2024-03-22T19:09:50.221505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230067+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:50.230067+0000","last_peered":"2024-03-22T19:09:50.230067+0000","last_clean":"2024-03-22T19:09:50.230067+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:50.230067+0000","last_undegraded":"2024-03-22T19:09:50.230067+0000","last_fullsized":"2024-03-22T19:09:50.230067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229198+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:50.229198+0000","last_peered":"2024-03-22T19:09:50.229198+0000","last_clean":"2024-03-22T19:09:50.229198+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:50.229198+0000","last_undegraded":"2024-03-22T19:09:50.229198+0000","last_fullsized":"2024-03-22T19:09:50.229198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224276+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:50.224276+0000","last_peered":"2024-03-22T19:09:50.224276+0000","last_clean":"2024-03-22T19:09:50.224276+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:50.224276+0000","last_undegraded":"2024-03-22T19:09:50.224276+0000","last_fullsized":"2024-03-22T19:09:50.224276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:02.177341+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:10:02.177341+0000","last_peered":"2024-03-22T19:10:02.177341+0000","last_clean":"2024-03-22T19:10:02.177341+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:10:02.177341+0000","last_undegraded":"2024-03-22T19:10:02.177341+0000","last_fullsized":"2024-03-22T19:10:02.177341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228903+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:50.228903+0000","last_peered":"2024-03-22T19:09:50.228903+0000","last_clean":"2024-03-22T19:09:50.228903+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:50.228903+0000","last_undegraded":"2024-03-22T19:09:50.228903+0000","last_fullsized":"2024-03-22T19:09:50.228903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.438753+0000","last_change":"2024-03-22T19:09:51.438753+0000","last_active":"2024-03-22T19:09:51.438753+0000","last_peered":"2024-03-22T19:09:51.438753+0000","last_clean":"2024-03-22T19:09:51.438753+0000","last_became_active":"2024-03-22T19:09:51.413301+0000","last_became_peered":"2024-03-22T19:09:51.413301+0000","last_unstale":"2024-03-22T19:09:51.438753+0000","last_undegraded":"2024-03-22T19:09:51.438753+0000","last_fullsized":"2024-03-22T19:09:51.438753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:49.994282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226106+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:50.226106+0000","last_peered":"2024-03-22T19:09:50.226106+0000","last_clean":"2024-03-22T19:09:50.226106+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:50.226106+0000","last_undegraded":"2024-03-22T19:09:50.226106+0000","last_fullsized":"2024-03-22T19:09:50.226106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232420+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:50.232420+0000","last_peered":"2024-03-22T19:09:50.232420+0000","last_clean":"2024-03-22T19:09:50.232420+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:50.232420+0000","last_undegraded":"2024-03-22T19:09:50.232420+0000","last_fullsized":"2024-03-22T19:09:50.232420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228700+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:50.228700+0000","last_peered":"2024-03-22T19:09:50.228700+0000","last_clean":"2024-03-22T19:09:50.228700+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:50.228700+0000","last_undegraded":"2024-03-22T19:09:50.228700+0000","last_fullsized":"2024-03-22T19:09:50.228700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'29","reported_seq":132,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:02.189799+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:10:02.189799+0000","last_peered":"2024-03-22T19:10:02.189799+0000","last_clean":"2024-03-22T19:10:02.189799+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:10:02.189799+0000","last_undegraded":"2024-03-22T19:10:02.189799+0000","last_fullsized":"2024-03-22T19:10:02.189799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":103,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230888+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:50.230888+0000","last_peered":"2024-03-22T19:09:50.230888+0000","last_clean":"2024-03-22T19:09:50.230888+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:50.230888+0000","last_undegraded":"2024-03-22T19:09:50.230888+0000","last_fullsized":"2024-03-22T19:09:50.230888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226121+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:50.226121+0000","last_peered":"2024-03-22T19:09:50.226121+0000","last_clean":"2024-03-22T19:09:50.226121+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:50.226121+0000","last_undegraded":"2024-03-22T19:09:50.226121+0000","last_fullsized":"2024-03-22T19:09:50.226121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221747+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:50.221747+0000","last_peered":"2024-03-22T19:09:50.221747+0000","last_clean":"2024-03-22T19:09:50.221747+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:50.221747+0000","last_undegraded":"2024-03-22T19:09:50.221747+0000","last_fullsized":"2024-03-22T19:09:50.221747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230590+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:50.230590+0000","last_peered":"2024-03-22T19:09:50.230590+0000","last_clean":"2024-03-22T19:09:50.230590+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:50.230590+0000","last_undegraded":"2024-03-22T19:09:50.230590+0000","last_fullsized":"2024-03-22T19:09:50.230590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224672+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:50.224672+0000","last_peered":"2024-03-22T19:09:50.224672+0000","last_clean":"2024-03-22T19:09:50.224672+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:50.224672+0000","last_undegraded":"2024-03-22T19:09:50.224672+0000","last_fullsized":"2024-03-22T19:09:50.224672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:58.172398+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:09:58.172398+0000","last_peered":"2024-03-22T19:09:58.172398+0000","last_clean":"2024-03-22T19:09:58.172398+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:09:58.172398+0000","last_undegraded":"2024-03-22T19:09:58.172398+0000","last_fullsized":"2024-03-22T19:09:58.172398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.435708+0000","last_change":"2024-03-22T19:09:51.435708+0000","last_active":"2024-03-22T19:09:51.435708+0000","last_peered":"2024-03-22T19:09:51.435708+0000","last_clean":"2024-03-22T19:09:51.435708+0000","last_became_active":"2024-03-22T19:09:51.412550+0000","last_became_peered":"2024-03-22T19:09:51.412550+0000","last_unstale":"2024-03-22T19:09:51.435708+0000","last_undegraded":"2024-03-22T19:09:51.435708+0000","last_fullsized":"2024-03-22T19:09:51.435708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:11:44.092847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229832+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:50.229832+0000","last_peered":"2024-03-22T19:09:50.229832+0000","last_clean":"2024-03-22T19:09:50.229832+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:50.229832+0000","last_undegraded":"2024-03-22T19:09:50.229832+0000","last_fullsized":"2024-03-22T19:09:50.229832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231493+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:50.231493+0000","last_peered":"2024-03-22T19:09:50.231493+0000","last_clean":"2024-03-22T19:09:50.231493+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:50.231493+0000","last_undegraded":"2024-03-22T19:09:50.231493+0000","last_fullsized":"2024-03-22T19:09:50.231493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229589+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:50.229589+0000","last_peered":"2024-03-22T19:09:50.229589+0000","last_clean":"2024-03-22T19:09:50.229589+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:50.229589+0000","last_undegraded":"2024-03-22T19:09:50.229589+0000","last_fullsized":"2024-03-22T19:09:50.229589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225716+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:50.225716+0000","last_peered":"2024-03-22T19:09:50.225716+0000","last_clean":"2024-03-22T19:09:50.225716+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:50.225716+0000","last_undegraded":"2024-03-22T19:09:50.225716+0000","last_fullsized":"2024-03-22T19:09:50.225716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227355+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:50.227355+0000","last_peered":"2024-03-22T19:09:50.227355+0000","last_clean":"2024-03-22T19:09:50.227355+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:50.227355+0000","last_undegraded":"2024-03-22T19:09:50.227355+0000","last_fullsized":"2024-03-22T19:09:50.227355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:00.174580+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:10:00.174580+0000","last_peered":"2024-03-22T19:10:00.174580+0000","last_clean":"2024-03-22T19:10:00.174580+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:10:00.174580+0000","last_undegraded":"2024-03-22T19:10:00.174580+0000","last_fullsized":"2024-03-22T19:10:00.174580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227464+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:50.227464+0000","last_peered":"2024-03-22T19:09:50.227464+0000","last_clean":"2024-03-22T19:09:50.227464+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:50.227464+0000","last_undegraded":"2024-03-22T19:09:50.227464+0000","last_fullsized":"2024-03-22T19:09:50.227464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229382+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:50.229382+0000","last_peered":"2024-03-22T19:09:50.229382+0000","last_clean":"2024-03-22T19:09:50.229382+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:50.229382+0000","last_undegraded":"2024-03-22T19:09:50.229382+0000","last_fullsized":"2024-03-22T19:09:50.229382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:58.171782+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:09:58.171782+0000","last_peered":"2024-03-22T19:09:58.171782+0000","last_clean":"2024-03-22T19:09:58.171782+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:09:58.171782+0000","last_undegraded":"2024-03-22T19:09:58.171782+0000","last_fullsized":"2024-03-22T19:09:58.171782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230745+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:50.230745+0000","last_peered":"2024-03-22T19:09:50.230745+0000","last_clean":"2024-03-22T19:09:50.230745+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:50.230745+0000","last_undegraded":"2024-03-22T19:09:50.230745+0000","last_fullsized":"2024-03-22T19:09:50.230745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223407+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:50.223407+0000","last_peered":"2024-03-22T19:09:50.223407+0000","last_clean":"2024-03-22T19:09:50.223407+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:50.223407+0000","last_undegraded":"2024-03-22T19:09:50.223407+0000","last_fullsized":"2024-03-22T19:09:50.223407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224404+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:50.224404+0000","last_peered":"2024-03-22T19:09:50.224404+0000","last_clean":"2024-03-22T19:09:50.224404+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:50.224404+0000","last_undegraded":"2024-03-22T19:09:50.224404+0000","last_fullsized":"2024-03-22T19:09:50.224404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221409+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:50.221409+0000","last_peered":"2024-03-22T19:09:50.221409+0000","last_clean":"2024-03-22T19:09:50.221409+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:50.221409+0000","last_undegraded":"2024-03-22T19:09:50.221409+0000","last_fullsized":"2024-03-22T19:09:50.221409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224573+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:50.224573+0000","last_peered":"2024-03-22T19:09:50.224573+0000","last_clean":"2024-03-22T19:09:50.224573+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:50.224573+0000","last_undegraded":"2024-03-22T19:09:50.224573+0000","last_fullsized":"2024-03-22T19:09:50.224573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231242+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:50.231242+0000","last_peered":"2024-03-22T19:09:50.231242+0000","last_clean":"2024-03-22T19:09:50.231242+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:50.231242+0000","last_undegraded":"2024-03-22T19:09:50.231242+0000","last_fullsized":"2024-03-22T19:09:50.231242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221846+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:50.221846+0000","last_peered":"2024-03-22T19:09:50.221846+0000","last_clean":"2024-03-22T19:09:50.221846+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:50.221846+0000","last_undegraded":"2024-03-22T19:09:50.221846+0000","last_fullsized":"2024-03-22T19:09:50.221846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224775+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:50.224775+0000","last_peered":"2024-03-22T19:09:50.224775+0000","last_clean":"2024-03-22T19:09:50.224775+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:50.224775+0000","last_undegraded":"2024-03-22T19:09:50.224775+0000","last_fullsized":"2024-03-22T19:09:50.224775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230722+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:50.230722+0000","last_peered":"2024-03-22T19:09:50.230722+0000","last_clean":"2024-03-22T19:09:50.230722+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:50.230722+0000","last_undegraded":"2024-03-22T19:09:50.230722+0000","last_fullsized":"2024-03-22T19:09:50.230722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223816+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:50.223816+0000","last_peered":"2024-03-22T19:09:50.223816+0000","last_clean":"2024-03-22T19:09:50.223816+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:50.223816+0000","last_undegraded":"2024-03-22T19:09:50.223816+0000","last_fullsized":"2024-03-22T19:09:50.223816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225947+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:50.225947+0000","last_peered":"2024-03-22T19:09:50.225947+0000","last_clean":"2024-03-22T19:09:50.225947+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:50.225947+0000","last_undegraded":"2024-03-22T19:09:50.225947+0000","last_fullsized":"2024-03-22T19:09:50.225947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228471+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:50.228471+0000","last_peered":"2024-03-22T19:09:50.228471+0000","last_clean":"2024-03-22T19:09:50.228471+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:50.228471+0000","last_undegraded":"2024-03-22T19:09:50.228471+0000","last_fullsized":"2024-03-22T19:09:50.228471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230778+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:50.230778+0000","last_peered":"2024-03-22T19:09:50.230778+0000","last_clean":"2024-03-22T19:09:50.230778+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:50.230778+0000","last_undegraded":"2024-03-22T19:09:50.230778+0000","last_fullsized":"2024-03-22T19:09:50.230778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230201+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:50.230201+0000","last_peered":"2024-03-22T19:09:50.230201+0000","last_clean":"2024-03-22T19:09:50.230201+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:50.230201+0000","last_undegraded":"2024-03-22T19:09:50.230201+0000","last_fullsized":"2024-03-22T19:09:50.230201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229252+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:50.229252+0000","last_peered":"2024-03-22T19:09:50.229252+0000","last_clean":"2024-03-22T19:09:50.229252+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:50.229252+0000","last_undegraded":"2024-03-22T19:09:50.229252+0000","last_fullsized":"2024-03-22T19:09:50.229252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.241586+0000","last_change":"2024-03-22T19:09:51.241586+0000","last_active":"2024-03-22T19:09:51.241586+0000","last_peered":"2024-03-22T19:09:51.241586+0000","last_clean":"2024-03-22T19:09:51.241586+0000","last_became_active":"2024-03-22T19:09:51.235327+0000","last_became_peered":"2024-03-22T19:09:51.235327+0000","last_unstale":"2024-03-22T19:09:51.241586+0000","last_undegraded":"2024-03-22T19:09:51.241586+0000","last_fullsized":"2024-03-22T19:09:51.241586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:43:59.307732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230257+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:50.230257+0000","last_peered":"2024-03-22T19:09:50.230257+0000","last_clean":"2024-03-22T19:09:50.230257+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:50.230257+0000","last_undegraded":"2024-03-22T19:09:50.230257+0000","last_fullsized":"2024-03-22T19:09:50.230257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231141+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:50.231141+0000","last_peered":"2024-03-22T19:09:50.231141+0000","last_clean":"2024-03-22T19:09:50.231141+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:50.231141+0000","last_undegraded":"2024-03-22T19:09:50.231141+0000","last_fullsized":"2024-03-22T19:09:50.231141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224983+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:50.224983+0000","last_peered":"2024-03-22T19:09:50.224983+0000","last_clean":"2024-03-22T19:09:50.224983+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:50.224983+0000","last_undegraded":"2024-03-22T19:09:50.224983+0000","last_fullsized":"2024-03-22T19:09:50.224983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:59.173923+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:09:59.173923+0000","last_peered":"2024-03-22T19:09:59.173923+0000","last_clean":"2024-03-22T19:09:59.173923+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:59.173923+0000","last_undegraded":"2024-03-22T19:09:59.173923+0000","last_fullsized":"2024-03-22T19:09:59.173923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224702+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:50.224702+0000","last_peered":"2024-03-22T19:09:50.224702+0000","last_clean":"2024-03-22T19:09:50.224702+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:50.224702+0000","last_undegraded":"2024-03-22T19:09:50.224702+0000","last_fullsized":"2024-03-22T19:09:50.224702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232307+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:50.232307+0000","last_peered":"2024-03-22T19:09:50.232307+0000","last_clean":"2024-03-22T19:09:50.232307+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:50.232307+0000","last_undegraded":"2024-03-22T19:09:50.232307+0000","last_fullsized":"2024-03-22T19:09:50.232307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221786+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:50.221786+0000","last_peered":"2024-03-22T19:09:50.221786+0000","last_clean":"2024-03-22T19:09:50.221786+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:50.221786+0000","last_undegraded":"2024-03-22T19:09:50.221786+0000","last_fullsized":"2024-03-22T19:09:50.221786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224337+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:50.224337+0000","last_peered":"2024-03-22T19:09:50.224337+0000","last_clean":"2024-03-22T19:09:50.224337+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:50.224337+0000","last_undegraded":"2024-03-22T19:09:50.224337+0000","last_fullsized":"2024-03-22T19:09:50.224337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229345+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:50.229345+0000","last_peered":"2024-03-22T19:09:50.229345+0000","last_clean":"2024-03-22T19:09:50.229345+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:50.229345+0000","last_undegraded":"2024-03-22T19:09:50.229345+0000","last_fullsized":"2024-03-22T19:09:50.229345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.412328+0000","last_change":"2024-03-22T19:09:51.412328+0000","last_active":"2024-03-22T19:09:51.412328+0000","last_peered":"2024-03-22T19:09:51.412328+0000","last_clean":"2024-03-22T19:09:51.412328+0000","last_became_active":"2024-03-22T19:09:51.411942+0000","last_became_peered":"2024-03-22T19:09:51.411942+0000","last_unstale":"2024-03-22T19:09:51.412328+0000","last_undegraded":"2024-03-22T19:09:51.412328+0000","last_fullsized":"2024-03-22T19:09:51.412328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:39:48.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225949+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:50.225949+0000","last_peered":"2024-03-22T19:09:50.225949+0000","last_clean":"2024-03-22T19:09:50.225949+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:50.225949+0000","last_undegraded":"2024-03-22T19:09:50.225949+0000","last_fullsized":"2024-03-22T19:09:50.225949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225646+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:50.225646+0000","last_peered":"2024-03-22T19:09:50.225646+0000","last_clean":"2024-03-22T19:09:50.225646+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:50.225646+0000","last_undegraded":"2024-03-22T19:09:50.225646+0000","last_fullsized":"2024-03-22T19:09:50.225646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280034+0000","last_change":"2024-03-22T19:09:51.280034+0000","last_active":"2024-03-22T19:09:51.280034+0000","last_peered":"2024-03-22T19:09:51.280034+0000","last_clean":"2024-03-22T19:09:51.280034+0000","last_became_active":"2024-03-22T19:09:51.279604+0000","last_became_peered":"2024-03-22T19:09:51.279604+0000","last_unstale":"2024-03-22T19:09:51.280034+0000","last_undegraded":"2024-03-22T19:09:51.280034+0000","last_fullsized":"2024-03-22T19:09:51.280034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:38:59.518086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229596+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:50.229596+0000","last_peered":"2024-03-22T19:09:50.229596+0000","last_clean":"2024-03-22T19:09:50.229596+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:50.229596+0000","last_undegraded":"2024-03-22T19:09:50.229596+0000","last_fullsized":"2024-03-22T19:09:50.229596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226600+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:50.226600+0000","last_peered":"2024-03-22T19:09:50.226600+0000","last_clean":"2024-03-22T19:09:50.226600+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:50.226600+0000","last_undegraded":"2024-03-22T19:09:50.226600+0000","last_fullsized":"2024-03-22T19:09:50.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230693+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:50.230693+0000","last_peered":"2024-03-22T19:09:50.230693+0000","last_clean":"2024-03-22T19:09:50.230693+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:50.230693+0000","last_undegraded":"2024-03-22T19:09:50.230693+0000","last_fullsized":"2024-03-22T19:09:50.230693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229186+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:50.229186+0000","last_peered":"2024-03-22T19:09:50.229186+0000","last_clean":"2024-03-22T19:09:50.229186+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:50.229186+0000","last_undegraded":"2024-03-22T19:09:50.229186+0000","last_fullsized":"2024-03-22T19:09:50.229186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230140+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:50.230140+0000","last_peered":"2024-03-22T19:09:50.230140+0000","last_clean":"2024-03-22T19:09:50.230140+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:50.230140+0000","last_undegraded":"2024-03-22T19:09:50.230140+0000","last_fullsized":"2024-03-22T19:09:50.230140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228785+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:50.228785+0000","last_peered":"2024-03-22T19:09:50.228785+0000","last_clean":"2024-03-22T19:09:50.228785+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:50.228785+0000","last_undegraded":"2024-03-22T19:09:50.228785+0000","last_fullsized":"2024-03-22T19:09:50.228785+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221650+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:50.221650+0000","last_peered":"2024-03-22T19:09:50.221650+0000","last_clean":"2024-03-22T19:09:50.221650+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:50.221650+0000","last_undegraded":"2024-03-22T19:09:50.221650+0000","last_fullsized":"2024-03-22T19:09:50.221650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229710+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:50.229710+0000","last_peered":"2024-03-22T19:09:50.229710+0000","last_clean":"2024-03-22T19:09:50.229710+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:50.229710+0000","last_undegraded":"2024-03-22T19:09:50.229710+0000","last_fullsized":"2024-03-22T19:09:50.229710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65969,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":442,"num_read_kb":295,"num_write":229,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":197907,"data_compressed":5466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":430423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.627},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77000000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.82699999999999996}]}]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080356,"data_compressed":37475,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.433}]},{"osd":2,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.626},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.498},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.875},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68400000000000005}]},{"osd":9,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.80600000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:09:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.753}]}]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":430289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.30599999999999999}]},{"osd":1,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.497}]},{"osd":2,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.625},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.44}]},{"osd":7,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri Mar 22 19:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.66100000000000003}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.68799999999999994}]}]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.377}]},{"osd":3,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.875,"15min":0.875},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.63500000000000001},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.622}]},{"osd":7,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66300000000000003}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1022761,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41799999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.60799999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.25700000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86699999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82899999999999996,"5min":1.111,"15min":1.111},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78600000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82399999999999995,"5min":0.878,"15min":0.878},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96399999999999997,"5min":1.155,"15min":1.155},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68200000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.878},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.80400000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83599999999999997}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77800000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72099999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.309}]},{"osd":1,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81699999999999995,"5min":0.875,"15min":0.875},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.369,"15min":1.369},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.76200000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.53400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71399999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78900000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.749,"15min":0.76100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84599999999999997,"5min":1.55,"15min":1.55},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85799999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Mar 22 19:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.502},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.75},"min":{"1min":0.53200000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:09:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.82299999999999995,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.68600000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.59499999999999997}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":489909,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66600000000000004,"15min":0.77700000000000002},"min":{"1min":0.42999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77700000000000002,"5min":6.71,"15min":6.71},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Mar 22 19:09:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Mar 22 19:09:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.502}]},{"osd":4,"last update":"Fri Mar 22 19:09:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.308,"5min":0.254,"15min":0.254},"max":{"1min":0.59699999999999998,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.45500000000000002,"15min":0.434},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80700000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.874,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Mar 22 19:09:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:09:27 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.752},{"interface":"front","average":{"1min":0.628,"5min":0.63,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82399999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65900000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.77200000000000002,"15min":0.752},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58899999999999997}]},{"osd":10,"last update":"Fri Mar 22 19:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.628},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":431297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:09:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:08:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45800000000000002,"15min":0.44400000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.748}]},{"osd":4,"last update":"Fri Mar 22 19:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64400000000000002,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96099999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":3.044},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":3.0590000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85599999999999998}]},{"osd":11,"last update":"Fri Mar 22 19:09:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":3.0790000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":489670,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.50600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:08:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.51500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.437},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65300000000000002,"5min":1.448,"15min":1.448},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Mar 22 19:09:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:09:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63500000000000001}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59999999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:09:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.625},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78000000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.42699999999999999},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.024,"5min":1.244,"15min":1.244},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.367,"15min":1.367},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56000000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84299999999999997,"5min":1.208,"15min":1.208},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.417,"15min":1.417},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97599999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Mar 22 19:09:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.077,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94299999999999995,"5min":1.641,"15min":1.641},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1220000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.749,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.74299999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953518,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.374,"15min":0.36199999999999999},"min":{"1min":0.29199999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38900000000000001,"15min":0.38},"min":{"1min":0.30399999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34599999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.626}]},{"osd":3,"last update":"Fri Mar 22 19:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82299999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83599999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.65600000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.495}]},{"osd":7,"last update":"Fri Mar 22 19:09:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.38,"5min":0.313,"15min":0.313},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.436,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92900000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.755}]},{"osd":9,"last update":"Fri Mar 22 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.73599999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.65700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1022485,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67600000000000005,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85799999999999998,"5min":3.887,"15min":3.887},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.66200000000000003,"15min":0.71299999999999997},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.376}]},{"osd":1,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47499999999999998}]},{"osd":2,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38,"15min":0.37},"min":{"1min":0.32800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.371},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.29599999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.433}]},{"osd":6,"last update":"Fri Mar 22 19:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.76300000000000001,"15min":0.78700000000000003},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83999999999999997,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 22 19:09:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59599999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:09:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Fri Mar 22 19:09:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61051,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61503,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61035,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:10:06.227 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-22T19:10:06.227 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-22T19:10:06.227 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-22T19:10:06.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-22T19:10:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:07.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:07 smithi086 ceph-mon[38032]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:07.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:07 smithi086 ceph-mon[38032]: pgmap v260: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-22T19:10:07.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:07 smithi131 ceph-mon[78559]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:07.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:07 smithi131 ceph-mon[78559]: pgmap v260: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-22T19:10:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:07 smithi167 ceph-mon[47728]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:07 smithi167 ceph-mon[47728]: pgmap v260: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-22T19:10:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 -- 172.21.15.86:0/2017055951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934100fb0 msgr2=0x7f7934101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 --2- 172.21.15.86:0/2017055951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934100fb0 0x7f7934101390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7928009990 tx=0x7f7928009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 -- 172.21.15.86:0/2017055951 shutdown_connections 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 --2- 172.21.15.86:0/2017055951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934106090 0x7f793410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 --2- 172.21.15.86:0/2017055951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934100fb0 0x7f7934101390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 --2- 172.21.15.86:0/2017055951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934101960 0x7f79341059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 -- 172.21.15.86:0/2017055951 >> 172.21.15.86:0/2017055951 conn(0x7f79340fc800 msgr2=0x7f79340fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 -- 172.21.15.86:0/2017055951 shutdown_connections 2024-03-22T19:10:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.881+0000 7f793b400700 1 -- 172.21.15.86:0/2017055951 wait complete. 2024-03-22T19:10:07.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.882+0000 7f793b400700 1 Processor -- start 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 -- start start 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 0x7f7934199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934101960 0x7f7934199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934106090 0x7f793419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793419e340 con 0x7f7934106090 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793419e4b0 con 0x7f7934101960 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f793b400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793419e670 con 0x7f7934100fb0 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f7938a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 0x7f7934199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f7939400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934106090 0x7f793419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f7938a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 0x7f7934199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:53276/0 (socket says 172.21.15.86:53276) 2024-03-22T19:10:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f7938a00700 1 -- 172.21.15.86:0/1845990171 learned_addr learned my addr 172.21.15.86:0/1845990171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.883+0000 7f7933e00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934101960 0x7f7934199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7938a00700 1 -- 172.21.15.86:0/1845990171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934101960 msgr2=0x7f7934199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7938a00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934101960 0x7f7934199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7938a00700 1 -- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934106090 msgr2=0x7f793419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7938a00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934106090 0x7f793419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7938a00700 1 -- 172.21.15.86:0/1845990171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280095f0 con 0x7f7934100fb0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7933e00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934101960 0x7f7934199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7938a00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 0x7f7934199440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f7928005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792801d070 con 0x7f7934100fb0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793419e8f0 con 0x7f7934100fb0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7928022470 con 0x7f7934100fb0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928017650 con 0x7f7934100fb0 2024-03-22T19:10:07.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.884+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79341a61b0 con 0x7f7934100fb0 2024-03-22T19:10:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.885+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7934102490 con 0x7f7934100fb0 2024-03-22T19:10:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.887+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f79280225e0 con 0x7f7934100fb0 2024-03-22T19:10:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.888+0000 7f7931600700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f791c0a0b20 0x7f791c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.889+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f79280cdef0 con 0x7f7934100fb0 2024-03-22T19:10:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.889+0000 7f7933e00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f791c0a0b20 0x7f791c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.889+0000 7f7933e00700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f791c0a0b20 0x7f791c0a2fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f793419a9a0 tx=0x7f7920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:07.891+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f79280977e0 con 0x7f7934100fb0 2024-03-22T19:10:08.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.134+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f793419a750 con 0x7f7934100fb0 2024-03-22T19:10:08.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.135+0000 7f7931600700 1 -- 172.21.15.86:0/1845990171 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f792809ae00 con 0x7f7934100fb0 2024-03-22T19:10:08.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:08.136 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T19:10:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f791c0a0b20 msgr2=0x7f791c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f791c0a0b20 0x7f791c0a2fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f793419a9a0 tx=0x7f7920009260 comp rx=0 tx=0).stop 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 msgr2=0x7f7934199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 0x7f7934199440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f7928005740 comp rx=0 tx=0).stop 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 shutdown_connections 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f791c0a0b20 0x7f791c0a2fe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7934106090 0x7f793419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7934100fb0 0x7f7934199440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 --2- 172.21.15.86:0/1845990171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7934101960 0x7f7934199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 >> 172.21.15.86:0/1845990171 conn(0x7f79340fc800 msgr2=0x7f79340feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 shutdown_connections 2024-03-22T19:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.137+0000 7f793b400700 1 -- 172.21.15.86:0/1845990171 wait complete. 2024-03-22T19:10:08.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:08 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1845990171' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T19:10:08.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:08 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1845990171' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T19:10:08.517 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-22T19:10:08.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:08.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:08 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1845990171' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T19:10:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 -- 172.21.15.86:0/1601931520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00073810 msgr2=0x7f8e00073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 --2- 172.21.15.86:0/1601931520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00073810 0x7f8e00073c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009990 tx=0x7f8dec009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 -- 172.21.15.86:0/1601931520 shutdown_connections 2024-03-22T19:10:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 --2- 172.21.15.86:0/1601931520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00108b10 0x7f8e0010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 --2- 172.21.15.86:0/1601931520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00074df0 0x7f8e00073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 --2- 172.21.15.86:0/1601931520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00073810 0x7f8e00073c90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 -- 172.21.15.86:0/1601931520 >> 172.21.15.86:0/1601931520 conn(0x7f8e000fc1c0 msgr2=0x7f8e000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 -- 172.21.15.86:0/1601931520 shutdown_connections 2024-03-22T19:10:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.713+0000 7f8e06200700 1 -- 172.21.15.86:0/1601931520 wait complete. 2024-03-22T19:10:08.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.714+0000 7f8e06200700 1 Processor -- start 2024-03-22T19:10:08.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 -- start start 2024-03-22T19:10:08.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00073810 0x7f8e00198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00074df0 0x7f8e00199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00108b10 0x7f8e001a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001a0960 con 0x7f8e00074df0 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001a0aa0 con 0x7f8e00108b10 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8e06200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001a0c30 con 0x7f8e00073810 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8dfea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00074df0 0x7f8e00199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8dff400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00073810 0x7f8e00198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:08.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8dff400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00073810 0x7f8e00198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36080/0 (socket says 172.21.15.86:36080) 2024-03-22T19:10:08.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.715+0000 7f8dff400700 1 -- 172.21.15.86:0/3849941412 learned_addr learned my addr 172.21.15.86:0/3849941412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:08.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dffe00700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00108b10 0x7f8e001a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:08.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dfea00700 1 -- 172.21.15.86:0/3849941412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00073810 msgr2=0x7f8e00198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dfea00700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00073810 0x7f8e00198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dfea00700 1 -- 172.21.15.86:0/3849941412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00108b10 msgr2=0x7f8e001a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dfea00700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00108b10 0x7f8e001a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dfea00700 1 -- 172.21.15.86:0/3849941412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec0095f0 con 0x7f8e00074df0 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8dfea00700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00074df0 0x7f8e00199370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8dec000c00 tx=0x7f8dec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec01d070 con 0x7f8e00074df0 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dec021410 con 0x7f8e00074df0 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e001a0eb0 con 0x7f8e00074df0 2024-03-22T19:10:08.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec017650 con 0x7f8e00074df0 2024-03-22T19:10:08.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.716+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e001a1220 con 0x7f8e00074df0 2024-03-22T19:10:08.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.717+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00067a50 con 0x7f8e00074df0 2024-03-22T19:10:08.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.719+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f8dec021580 con 0x7f8e00074df0 2024-03-22T19:10:08.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.720+0000 7f8df7e00700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8de40a0b20 0x7f8de40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:08.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.721+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8dec0cd550 con 0x7f8e00074df0 2024-03-22T19:10:08.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.721+0000 7f8dff400700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8de40a0b20 0x7f8de40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:08.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.721+0000 7f8dff400700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8de40a0b20 0x7f8de40a2fe0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009d20 tx=0x7f8df0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:08.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.724+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8e00067a50 con 0x7f8e00074df0 2024-03-22T19:10:08.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.966+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e001078d0 con 0x7f8e00074df0 2024-03-22T19:10:08.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.968+0000 7f8df7e00700 1 -- 172.21.15.86:0/3849941412 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7f8dec096e40 con 0x7f8e00074df0 2024-03-22T19:10:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:08.970 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8de40a0b20 msgr2=0x7f8de40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8de40a0b20 0x7f8de40a2fe0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009d20 tx=0x7f8df0009260 comp rx=0 tx=0).stop 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00074df0 msgr2=0x7f8e00199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00074df0 0x7f8e00199370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8dec000c00 tx=0x7f8dec009580 comp rx=0 tx=0).stop 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 shutdown_connections 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8de40a0b20 0x7f8de40a2fe0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00108b10 0x7f8e001a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e00073810 0x7f8e00198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 --2- 172.21.15.86:0/3849941412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e00074df0 0x7f8e00199370 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.970+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 >> 172.21.15.86:0/3849941412 conn(0x7f8e000fc1c0 msgr2=0x7f8e00104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.971+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 shutdown_connections 2024-03-22T19:10:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:08.971+0000 7f8e06200700 1 -- 172.21.15.86:0/3849941412 wait complete. 2024-03-22T19:10:08.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:08.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:09.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 -- 172.21.15.86:0/662295821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 msgr2=0x7f143c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 --2- 172.21.15.86:0/662295821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 0x7f143c102390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14280099c0 tx=0x7f1428009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 -- 172.21.15.86:0/662295821 shutdown_connections 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 --2- 172.21.15.86:0/662295821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f143c1028d0 0x7f143c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 --2- 172.21.15.86:0/662295821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 --2- 172.21.15.86:0/662295821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 0x7f143c102390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 -- 172.21.15.86:0/662295821 >> 172.21.15.86:0/662295821 conn(0x7f143c0fc2b0 msgr2=0x7f143c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 -- 172.21.15.86:0/662295821 shutdown_connections 2024-03-22T19:10:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.186+0000 7f1442400700 1 -- 172.21.15.86:0/662295821 wait complete. 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 Processor -- start 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 -- start start 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 0x7f143c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f143c1028d0 0x7f143c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c19ab50 con 0x7f143c100d40 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c073c20 con 0x7f143c101f30 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f1442400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c073de0 con 0x7f143c1028d0 2024-03-22T19:10:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.187+0000 7f143b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41450/0 (socket says 172.21.15.86:41450) 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 -- 172.21.15.86:0/3086154001 learned_addr learned my addr 172.21.15.86:0/3086154001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143be00700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f143c1028d0 0x7f143c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143aa00700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 0x7f143c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 -- 172.21.15.86:0/3086154001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f143c1028d0 msgr2=0x7f143c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f143c1028d0 0x7f143c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 -- 172.21.15.86:0/3086154001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 msgr2=0x7f143c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 0x7f143c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 -- 172.21.15.86:0/3086154001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280095f0 con 0x7f143c100d40 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f143b400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c074b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e970 tx=0x7f142c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c009790 con 0x7f143c100d40 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f142c00c7e0 con 0x7f143c100d40 2024-03-22T19:10:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c1a5500 con 0x7f143c100d40 2024-03-22T19:10:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.188+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c003c70 con 0x7f143c100d40 2024-03-22T19:10:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.189+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c1a58c0 con 0x7f143c100d40 2024-03-22T19:10:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.190+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c108210 con 0x7f143c100d40 2024-03-22T19:10:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.192+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f142c00cc50 con 0x7f143c100d40 2024-03-22T19:10:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.193+0000 7f1433e00700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14200a0b20 0x7f14200a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.193+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f142c014070 con 0x7f143c100d40 2024-03-22T19:10:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.193+0000 7f143aa00700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14200a0b20 0x7f14200a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.194+0000 7f143aa00700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14200a0b20 0x7f14200a2fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1428009990 tx=0x7f1428000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.196+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f142c095c20 con 0x7f143c100d40 2024-03-22T19:10:09.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:09 smithi086 ceph-mon[38032]: pgmap v261: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:09.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:09 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3849941412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:09.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.402+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f143c02ccb0 con 0x7f143c100d40 2024-03-22T19:10:09.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.402+0000 7f1433e00700 1 -- 172.21.15.86:0/3086154001 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f142c003dd0 con 0x7f143c100d40 2024-03-22T19:10:09.404 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14200a0b20 msgr2=0x7f14200a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14200a0b20 0x7f14200a2fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1428009990 tx=0x7f1428000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 msgr2=0x7f143c074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c074b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e970 tx=0x7f142c00ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 shutdown_connections 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14200a0b20 0x7f14200a2fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f143c1028d0 0x7f143c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c100d40 0x7f143c074b10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 --2- 172.21.15.86:0/3086154001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f143c101f30 0x7f143c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 >> 172.21.15.86:0/3086154001 conn(0x7f143c0fc2b0 msgr2=0x7f143c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:09.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 shutdown_connections 2024-03-22T19:10:09.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.405+0000 7f1442400700 1 -- 172.21.15.86:0/3086154001 wait complete. 2024-03-22T19:10:09.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:09.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:09 smithi131 ceph-mon[78559]: pgmap v261: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:09.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:09 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3849941412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:09 smithi167 ceph-mon[47728]: pgmap v261: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:09 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3849941412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 -- 172.21.15.86:0/1844821567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 msgr2=0x7fa618101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 --2- 172.21.15.86:0/1844821567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618101160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa6080099c0 tx=0x7fa608009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 -- 172.21.15.86:0/1844821567 shutdown_connections 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 --2- 172.21.15.86:0/1844821567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6181028d0 0x7fa618109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 --2- 172.21.15.86:0/1844821567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618101160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 --2- 172.21.15.86:0/1844821567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa618101f30 0x7fa618102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 -- 172.21.15.86:0/1844821567 >> 172.21.15.86:0/1844821567 conn(0x7fa6180fc2b0 msgr2=0x7fa6180fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 -- 172.21.15.86:0/1844821567 shutdown_connections 2024-03-22T19:10:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.619+0000 7fa61ec00700 1 -- 172.21.15.86:0/1844821567 wait complete. 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.620+0000 7fa61ec00700 1 Processor -- start 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 -- start start 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa618101f30 0x7fa6180731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6181028d0 0x7fa6180736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618073de0 con 0x7fa618100d40 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618073f50 con 0x7fa6181028d0 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa61ec00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6181a0d80 con 0x7fa618101f30 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa617e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa617e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41468/0 (socket says 172.21.15.86:41468) 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa617e00700 1 -- 172.21.15.86:0/555264357 learned_addr learned my addr 172.21.15.86:0/555264357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.621+0000 7fa617400700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa618101f30 0x7fa6180731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa61cc00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6181028d0 0x7fa6180736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa617e00700 1 -- 172.21.15.86:0/555264357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa618101f30 msgr2=0x7fa6180731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa617e00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa618101f30 0x7fa6180731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa617e00700 1 -- 172.21.15.86:0/555264357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6181028d0 msgr2=0x7fa6180736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa617e00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6181028d0 0x7fa6180736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa617e00700 1 -- 172.21.15.86:0/555264357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6080095f0 con 0x7fa618100d40 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa617e00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618074ac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa608003730 tx=0x7fa60800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60801d070 con 0x7fa618100d40 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa60800fe70 con 0x7fa618100d40 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6181a0f80 con 0x7fa618100d40 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.622+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080173f0 con 0x7fa618100d40 2024-03-22T19:10:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.623+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6181a1340 con 0x7fa618100d40 2024-03-22T19:10:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.624+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa618108210 con 0x7fa618100d40 2024-03-22T19:10:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.625+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fa608022470 con 0x7fa618100d40 2024-03-22T19:10:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.626+0000 7fa614c00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa5fc0a0ba0 0x7fa5fc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.627+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa6080cd330 con 0x7fa618100d40 2024-03-22T19:10:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.627+0000 7fa617400700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa5fc0a0ba0 0x7fa5fc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.627+0000 7fa617400700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa5fc0a0ba0 0x7fa5fc0a3060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa604009c60 tx=0x7fa604009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:09.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.630+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa608096c20 con 0x7fa618100d40 2024-03-22T19:10:09.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.832+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa61802cdc0 con 0x7fa618100d40 2024-03-22T19:10:09.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.833+0000 7fa614c00700 1 -- 172.21.15.86:0/555264357 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa60809a240 con 0x7fa618100d40 2024-03-22T19:10:09.834 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa5fc0a0ba0 msgr2=0x7fa5fc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa5fc0a0ba0 0x7fa5fc0a3060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa604009c60 tx=0x7fa604009260 comp rx=0 tx=0).stop 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 msgr2=0x7fa618074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618074ac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa608003730 tx=0x7fa60800fa00 comp rx=0 tx=0).stop 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 shutdown_connections 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa5fc0a0ba0 0x7fa5fc0a3060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6181028d0 0x7fa6180736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa618100d40 0x7fa618074ac0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 --2- 172.21.15.86:0/555264357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa618101f30 0x7fa6180731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 >> 172.21.15.86:0/555264357 conn(0x7fa6180fc2b0 msgr2=0x7fa618105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 shutdown_connections 2024-03-22T19:10:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:09.835+0000 7fa61ec00700 1 -- 172.21.15.86:0/555264357 wait complete. 2024-03-22T19:10:09.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 -- 172.21.15.86:0/3857448296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290100d40 msgr2=0x7fc290101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 --2- 172.21.15.86:0/3857448296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290100d40 0x7fc290101160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2840099c0 tx=0x7fc284009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 -- 172.21.15.86:0/3857448296 shutdown_connections 2024-03-22T19:10:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 --2- 172.21.15.86:0/3857448296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 0x7fc290109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 --2- 172.21.15.86:0/3857448296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290100d40 0x7fc290101160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 --2- 172.21.15.86:0/3857448296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc290101f30 0x7fc290102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 -- 172.21.15.86:0/3857448296 >> 172.21.15.86:0/3857448296 conn(0x7fc2900fc2b0 msgr2=0x7fc2900fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 -- 172.21.15.86:0/3857448296 shutdown_connections 2024-03-22T19:10:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.048+0000 7fc297e00700 1 -- 172.21.15.86:0/3857448296 wait complete. 2024-03-22T19:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.049+0000 7fc297e00700 1 Processor -- start 2024-03-22T19:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.049+0000 7fc297e00700 1 -- start start 2024-03-22T19:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc297e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc290100d40 0x7fc290198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc297e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 0x7fc290199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc297e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 0x7fc2901a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc297e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2901a0a40 con 0x7fc290101f30 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc297e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2901a0b80 con 0x7fc2901028d0 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc297e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2901a0d10 con 0x7fc290100d40 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 0x7fc290199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 0x7fc290199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41486/0 (socket says 172.21.15.86:41486) 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 -- 172.21.15.86:0/559472420 learned_addr learned my addr 172.21.15.86:0/559472420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc295400700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc290100d40 0x7fc290198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc295e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 0x7fc2901a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 -- 172.21.15.86:0/559472420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc290100d40 msgr2=0x7fc290198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc290100d40 0x7fc290198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 -- 172.21.15.86:0/559472420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 msgr2=0x7fc2901a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 0x7fc2901a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc294a00700 1 -- 172.21.15.86:0/559472420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840095f0 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.050+0000 7fc295e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 0x7fc2901a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.051+0000 7fc294a00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 0x7fc290199450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00e9a0 tx=0x7fc27c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.051+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c009790 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.051+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27c00cbd0 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.051+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2901a0ff0 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.051+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c018680 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.051+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2901a14c0 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.052+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc290067ac0 con 0x7fc290101f30 2024-03-22T19:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.054+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc27c00cd40 con 0x7fc290101f30 2024-03-22T19:10:10.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.055+0000 7fc28e000700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2780a0ba0 0x7fc2780a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.055+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc27c014070 con 0x7fc290101f30 2024-03-22T19:10:10.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.055+0000 7fc295400700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2780a0ba0 0x7fc2780a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.056+0000 7fc295400700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2780a0ba0 0x7fc2780a3060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc284003730 tx=0x7fc284000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:10.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.059+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc27c0529f0 con 0x7fc290101f30 2024-03-22T19:10:10.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.271+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2901a1810 con 0x7fc290101f30 2024-03-22T19:10:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.271+0000 7fc28e000700 1 -- 172.21.15.86:0/559472420 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc27c095c10 con 0x7fc290101f30 2024-03-22T19:10:10.273 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:10.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2780a0ba0 msgr2=0x7fc2780a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2780a0ba0 0x7fc2780a3060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc284003730 tx=0x7fc284000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 msgr2=0x7fc290199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 0x7fc290199450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00e9a0 tx=0x7fc27c00ecb0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 shutdown_connections 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2780a0ba0 0x7fc2780a3060 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2901028d0 0x7fc2901a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc290100d40 0x7fc290198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 --2- 172.21.15.86:0/559472420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc290101f30 0x7fc290199450 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 >> 172.21.15.86:0/559472420 conn(0x7fc2900fc2b0 msgr2=0x7fc2901076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 shutdown_connections 2024-03-22T19:10:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.274+0000 7fc297e00700 1 -- 172.21.15.86:0/559472420 wait complete. 2024-03-22T19:10:10.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 -- 172.21.15.86:0/567442130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 msgr2=0x7f9cf8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 --2- 172.21.15.86:0/567442130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8073240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009990 tx=0x7f9ce8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 -- 172.21.15.86:0/567442130 shutdown_connections 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 --2- 172.21.15.86:0/567442130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cf8108b10 0x7f9cf810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 --2- 172.21.15.86:0/567442130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8073240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 --2- 172.21.15.86:0/567442130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 0x7f9cf8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 -- 172.21.15.86:0/567442130 >> 172.21.15.86:0/567442130 conn(0x7f9cf80fc1c0 msgr2=0x7f9cf80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 -- 172.21.15.86:0/567442130 shutdown_connections 2024-03-22T19:10:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.485+0000 7f9cfe800700 1 -- 172.21.15.86:0/567442130 wait complete. 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.486+0000 7f9cfe800700 1 Processor -- start 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 -- start start 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 0x7f9cf8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cf8108b10 0x7f9cf81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81a08f0 con 0x7f9cf8074df0 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81a0a60 con 0x7f9cf8108b10 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cfe800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81a0c20 con 0x7f9cf8073810 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cf6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cf7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 0x7f9cf8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cf6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41504/0 (socket says 172.21.15.86:41504) 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cf6a00700 1 -- 172.21.15.86:0/1953421375 learned_addr learned my addr 172.21.15.86:0/1953421375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.487+0000 7f9cf7e00700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cf8108b10 0x7f9cf81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf7400700 1 -- 172.21.15.86:0/1953421375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cf8108b10 msgr2=0x7f9cf81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf7400700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cf8108b10 0x7f9cf81a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf7400700 1 -- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 msgr2=0x7f9cf8199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf7400700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf7400700 1 -- 172.21.15.86:0/1953421375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80095f0 con 0x7f9cf8073810 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf6a00700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cf7400700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 0x7f9cf8198e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8000c00 tx=0x7f9ce8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce801d070 con 0x7f9cf8073810 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce8021410 con 0x7f9cf8073810 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf81a0ea0 con 0x7f9cf8073810 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8017650 con 0x7f9cf8073810 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.488+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf81a1210 con 0x7f9cf8073810 2024-03-22T19:10:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.489+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf81078d0 con 0x7f9cf8073810 2024-03-22T19:10:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.491+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f9ce8021580 con 0x7f9cf8073810 2024-03-22T19:10:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.492+0000 7f9cefe00700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9cdc0a0b20 0x7f9cdc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.492+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9ce80cd340 con 0x7f9cf8073810 2024-03-22T19:10:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.492+0000 7f9cf6a00700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9cdc0a0b20 0x7f9cdc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.493+0000 7f9cf6a00700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9cdc0a0b20 0x7f9cdc0a2fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007fd0 tx=0x7f9ce4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:10.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.498+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9ce8096d30 con 0x7f9cf8073810 2024-03-22T19:10:10.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3086154001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:10.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/555264357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:10 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3086154001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:10 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/555264357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:10 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3086154001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:10 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/555264357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.745+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cf80679e0 con 0x7f9cf8073810 2024-03-22T19:10:10.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.747+0000 7f9cefe00700 1 -- 172.21.15.86:0/1953421375 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f9ce809a350 con 0x7f9cf8073810 2024-03-22T19:10:10.748 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:10.748 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:09:21.111792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9cdc0a0b20 msgr2=0x7f9cdc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9cdc0a0b20 0x7f9cdc0a2fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007fd0 tx=0x7f9ce4007480 comp rx=0 tx=0).stop 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 msgr2=0x7f9cf8198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 0x7f9cf8198e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8000c00 tx=0x7f9ce8005740 comp rx=0 tx=0).stop 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 shutdown_connections 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9cdc0a0b20 0x7f9cdc0a2fe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cf8108b10 0x7f9cf81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8073810 0x7f9cf8198e00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 --2- 172.21.15.86:0/1953421375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf8074df0 0x7f9cf8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 >> 172.21.15.86:0/1953421375 conn(0x7f9cf80fc1c0 msgr2=0x7f9cf8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 shutdown_connections 2024-03-22T19:10:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.749+0000 7f9cfe800700 1 -- 172.21.15.86:0/1953421375 wait complete. 2024-03-22T19:10:10.752 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:10:10.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:10.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 -- 172.21.15.86:0/608439548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320073810 msgr2=0x7f1320073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 --2- 172.21.15.86:0/608439548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320073810 0x7f1320073c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f130c0098f0 tx=0x7f130c009c00 comp rx=0 tx=0).stop 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 -- 172.21.15.86:0/608439548 shutdown_connections 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 --2- 172.21.15.86:0/608439548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320108b10 0x7f132010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 --2- 172.21.15.86:0/608439548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320074df0 0x7f1320073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 --2- 172.21.15.86:0/608439548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320073810 0x7f1320073c90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 -- 172.21.15.86:0/608439548 >> 172.21.15.86:0/608439548 conn(0x7f13200fc1c0 msgr2=0x7f13200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 -- 172.21.15.86:0/608439548 shutdown_connections 2024-03-22T19:10:10.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.960+0000 7f1328600700 1 -- 172.21.15.86:0/608439548 wait complete. 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 Processor -- start 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 -- start start 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 0x7f1320198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320074df0 0x7f13201993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320108b10 0x7f13201a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13201a09a0 con 0x7f1320073810 2024-03-22T19:10:10.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13201a0ae0 con 0x7f1320108b10 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1328600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13201a0c70 con 0x7f1320074df0 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.961+0000 7f1325c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 0x7f1320198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 0x7f1320198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41524/0 (socket says 172.21.15.86:41524) 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1326600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320108b10 0x7f13201a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 -- 172.21.15.86:0/363157644 learned_addr learned my addr 172.21.15.86:0/363157644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325200700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320074df0 0x7f13201993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 -- 172.21.15.86:0/363157644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320074df0 msgr2=0x7f13201993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320074df0 0x7f13201993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 -- 172.21.15.86:0/363157644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320108b10 msgr2=0x7f13201a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320108b10 0x7f13201a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 -- 172.21.15.86:0/363157644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c009520 con 0x7f1320073810 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1326600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320108b10 0x7f13201a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1325c00700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 0x7f1320198e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f131400e9a0 tx=0x7f131400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314009790 con 0x7f1320073810 2024-03-22T19:10:10.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1314010430 con 0x7f1320073810 2024-03-22T19:10:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13201a0f50 con 0x7f1320073810 2024-03-22T19:10:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.962+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314018680 con 0x7f1320073810 2024-03-22T19:10:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.963+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13201a1370 con 0x7f1320073810 2024-03-22T19:10:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.964+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13201078d0 con 0x7f1320073810 2024-03-22T19:10:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.965+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f13140105a0 con 0x7f1320073810 2024-03-22T19:10:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.967+0000 7f131e000700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13080a0b20 0x7f13080a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.967+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1314014070 con 0x7f1320073810 2024-03-22T19:10:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.967+0000 7f1325200700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13080a0b20 0x7f13080a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:10.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.967+0000 7f1325200700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13080a0b20 0x7f13080a2fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f130c0098c0 tx=0x7f130c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:10.970+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f13140529f0 con 0x7f1320073810 2024-03-22T19:10:11.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.211+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13200679e0 con 0x7f1320073810 2024-03-22T19:10:11.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.213+0000 7f131e000700 1 -- 172.21.15.86:0/363157644 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7f1314095bf0 con 0x7f1320073810 2024-03-22T19:10:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:11.214 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:11.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13080a0b20 msgr2=0x7f13080a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13080a0b20 0x7f13080a2fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f130c0098c0 tx=0x7f130c019040 comp rx=0 tx=0).stop 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 msgr2=0x7f1320198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 0x7f1320198e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f131400e9a0 tx=0x7f131400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 shutdown_connections 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13080a0b20 0x7f13080a2fe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320108b10 0x7f13201a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320073810 0x7f1320198e70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 --2- 172.21.15.86:0/363157644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320074df0 0x7f13201993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 >> 172.21.15.86:0/363157644 conn(0x7f13200fc1c0 msgr2=0x7f1320104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 shutdown_connections 2024-03-22T19:10:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.215+0000 7f1328600700 1 -- 172.21.15.86:0/363157644 wait complete. 2024-03-22T19:10:11.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:11.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 -- 172.21.15.86:0/393070052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c1028e0 msgr2=0x7f073c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 --2- 172.21.15.86:0/393070052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c1028e0 0x7f073c109190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f073400b0d0 tx=0x7f073400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 -- 172.21.15.86:0/393070052 shutdown_connections 2024-03-22T19:10:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 --2- 172.21.15.86:0/393070052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c1028e0 0x7f073c109190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 --2- 172.21.15.86:0/393070052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f073c100d50 0x7f073c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 --2- 172.21.15.86:0/393070052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c101f40 0x7f073c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 -- 172.21.15.86:0/393070052 >> 172.21.15.86:0/393070052 conn(0x7f073c0fc2c0 msgr2=0x7f073c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.431+0000 7f0742000700 1 -- 172.21.15.86:0/393070052 shutdown_connections 2024-03-22T19:10:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.432+0000 7f0742000700 1 -- 172.21.15.86:0/393070052 wait complete. 2024-03-22T19:10:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 Processor -- start 2024-03-22T19:10:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 -- start start 2024-03-22T19:10:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 0x7f073c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f073c101f40 0x7f073c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c1028e0 0x7f073c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1a0b50 con 0x7f073c100d50 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1a0c90 con 0x7f073c1028e0 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f0742000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1a0e20 con 0x7f073c101f40 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f073b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 0x7f073c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f073b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 0x7f073c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41552/0 (socket says 172.21.15.86:41552) 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f073b400700 1 -- 172.21.15.86:0/2976037238 learned_addr learned my addr 172.21.15.86:0/2976037238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f073aa00700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f073c101f40 0x7f073c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.433+0000 7f073be00700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c1028e0 0x7f073c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073b400700 1 -- 172.21.15.86:0/2976037238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f073c101f40 msgr2=0x7f073c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073b400700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f073c101f40 0x7f073c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073b400700 1 -- 172.21.15.86:0/2976037238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c1028e0 msgr2=0x7f073c1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073b400700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c1028e0 0x7f073c1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073b400700 1 -- 172.21.15.86:0/2976037238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734009d00 con 0x7f073c100d50 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073be00700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c1028e0 0x7f073c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f073b400700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 0x7f073c199020 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f072c00e970 tx=0x7f072c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c009790 con 0x7f073c100d50 2024-03-22T19:10:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f072c00eec0 con 0x7f073c100d50 2024-03-22T19:10:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c1a1100 con 0x7f073c100d50 2024-03-22T19:10:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c003cf0 con 0x7f073c100d50 2024-03-22T19:10:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.434+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1a1660 con 0x7f073c100d50 2024-03-22T19:10:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.435+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c067ac0 con 0x7f073c100d50 2024-03-22T19:10:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.437+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f072c00ccd0 con 0x7f073c100d50 2024-03-22T19:10:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.439+0000 7f0733e00700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f07200a0ba0 0x7f07200a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.439+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f072c014070 con 0x7f073c100d50 2024-03-22T19:10:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.439+0000 7f073aa00700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f07200a0ba0 0x7f07200a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.440+0000 7f073aa00700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f07200a0ba0 0x7f07200a3060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0728009c60 tx=0x7f0728009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.443+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f072c095c20 con 0x7f073c100d50 2024-03-22T19:10:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:11 smithi086 ceph-mon[38032]: pgmap v262: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:11 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/559472420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:11 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1953421375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:11 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/363157644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:11 smithi167 ceph-mon[47728]: pgmap v262: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:11 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/559472420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:11 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1953421375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:11 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/363157644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.649+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f073c1a18e0 con 0x7f073c100d50 2024-03-22T19:10:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.650+0000 7f0733e00700 1 -- 172.21.15.86:0/2976037238 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f072c099240 con 0x7f073c100d50 2024-03-22T19:10:11.650 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f07200a0ba0 msgr2=0x7f07200a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f07200a0ba0 0x7f07200a3060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0728009c60 tx=0x7f0728009260 comp rx=0 tx=0).stop 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 msgr2=0x7f073c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 0x7f073c199020 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f072c00e970 tx=0x7f072c00ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 shutdown_connections 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f07200a0ba0 0x7f07200a3060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f073c1028e0 0x7f073c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c100d50 0x7f073c199020 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 --2- 172.21.15.86:0/2976037238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f073c101f40 0x7f073c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 >> 172.21.15.86:0/2976037238 conn(0x7f073c0fc2c0 msgr2=0x7f073c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 shutdown_connections 2024-03-22T19:10:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.652+0000 7f0742000700 1 -- 172.21.15.86:0/2976037238 wait complete. 2024-03-22T19:10:11.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:11 smithi131 ceph-mon[78559]: pgmap v262: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:11 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/559472420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:11 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1953421375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:11 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/363157644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:11.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 -- 172.21.15.86:0/622809373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c1028e0 msgr2=0x7f784c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 --2- 172.21.15.86:0/622809373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c1028e0 0x7f784c109190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f784800b0d0 tx=0x7f784800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 -- 172.21.15.86:0/622809373 shutdown_connections 2024-03-22T19:10:11.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 --2- 172.21.15.86:0/622809373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c1028e0 0x7f784c109190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 --2- 172.21.15.86:0/622809373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c100d50 0x7f784c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 --2- 172.21.15.86:0/622809373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c101f40 0x7f784c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 -- 172.21.15.86:0/622809373 >> 172.21.15.86:0/622809373 conn(0x7f784c0fc2c0 msgr2=0x7f784c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:11.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.865+0000 7f7854800700 1 -- 172.21.15.86:0/622809373 shutdown_connections 2024-03-22T19:10:11.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.866+0000 7f7854800700 1 -- 172.21.15.86:0/622809373 wait complete. 2024-03-22T19:10:11.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.867+0000 7f7854800700 1 Processor -- start 2024-03-22T19:10:11.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.867+0000 7f7854800700 1 -- start start 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7854800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c100d50 0x7f784c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7854800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 0x7f784c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7854800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c1028e0 0x7f784c1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7854800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1a0b20 con 0x7f784c101f40 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7854800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1a0c60 con 0x7f784c100d50 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7854800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1a0df0 con 0x7f784c1028e0 2024-03-22T19:10:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 0x7f784c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 0x7f784c199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41564/0 (socket says 172.21.15.86:41564) 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 -- 172.21.15.86:0/2424256584 learned_addr learned my addr 172.21.15.86:0/2424256584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851e00700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c100d50 0x7f784c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7852800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c1028e0 0x7f784c1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 -- 172.21.15.86:0/2424256584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c1028e0 msgr2=0x7f784c1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c1028e0 0x7f784c1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 -- 172.21.15.86:0/2424256584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c100d50 msgr2=0x7f784c198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c100d50 0x7f784c198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 -- 172.21.15.86:0/2424256584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009520 con 0x7f784c101f40 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.868+0000 7f7851400700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 0x7f784c199530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f783c00e9a0 tx=0x7f783c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.869+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c009790 con 0x7f784c101f40 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.869+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783c010430 con 0x7f784c101f40 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.869+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c018680 con 0x7f784c101f40 2024-03-22T19:10:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.869+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7848009d00 con 0x7f784c101f40 2024-03-22T19:10:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.869+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c1a1400 con 0x7f784c101f40 2024-03-22T19:10:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.870+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c067ac0 con 0x7f784c101f40 2024-03-22T19:10:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.872+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f783c0105a0 con 0x7f784c101f40 2024-03-22T19:10:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.874+0000 7f7846a00700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78340a0ba0 0x7f78340a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.874+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f783c014070 con 0x7f784c101f40 2024-03-22T19:10:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.874+0000 7f7851e00700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78340a0ba0 0x7f78340a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.874+0000 7f7851e00700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78340a0ba0 0x7f78340a3060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f784c101da0 tx=0x7f7840009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:11.877+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f783c0529f0 con 0x7f784c101f40 2024-03-22T19:10:12.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.082+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f784c1a1620 con 0x7f784c101f40 2024-03-22T19:10:12.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.083+0000 7f7846a00700 1 -- 172.21.15.86:0/2424256584 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f783c095b70 con 0x7f784c101f40 2024-03-22T19:10:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.085+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78340a0ba0 msgr2=0x7f78340a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.085+0000 7f7854800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78340a0ba0 0x7f78340a3060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f784c101da0 tx=0x7f7840009260 comp rx=0 tx=0).stop 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.085+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 msgr2=0x7f784c199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.085+0000 7f7854800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 0x7f784c199530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f783c00e9a0 tx=0x7f783c00ecb0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 shutdown_connections 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f78340a0ba0 0x7f78340a3060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c1028e0 0x7f784c1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c100d50 0x7f784c198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 --2- 172.21.15.86:0/2424256584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c101f40 0x7f784c199530 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 >> 172.21.15.86:0/2424256584 conn(0x7f784c0fc2c0 msgr2=0x7f784c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 shutdown_connections 2024-03-22T19:10:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.086+0000 7f7854800700 1 -- 172.21.15.86:0/2424256584 wait complete. 2024-03-22T19:10:12.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.302+0000 7f51b7800700 1 -- 172.21.15.86:0/2586239351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 msgr2=0x7f51b01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.302+0000 7f51b7800700 1 --2- 172.21.15.86:0/2586239351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b01023a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f519c0099c0 tx=0x7f519c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 -- 172.21.15.86:0/2586239351 shutdown_connections 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 --2- 172.21.15.86:0/2586239351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b01028e0 0x7f51b0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 --2- 172.21.15.86:0/2586239351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51b0100d50 0x7f51b0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 --2- 172.21.15.86:0/2586239351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b01023a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 -- 172.21.15.86:0/2586239351 >> 172.21.15.86:0/2586239351 conn(0x7f51b00fc2c0 msgr2=0x7f51b00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 -- 172.21.15.86:0/2586239351 shutdown_connections 2024-03-22T19:10:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.303+0000 7f51b7800700 1 -- 172.21.15.86:0/2586239351 wait complete. 2024-03-22T19:10:12.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 Processor -- start 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 -- start start 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b0100d50 0x7f51b0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51b01028e0 0x7f51b00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0073e80 con 0x7f51b0101f40 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0073ff0 con 0x7f51b01028e0 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51b7800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01a0d30 con 0x7f51b0100d50 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41588/0 (socket says 172.21.15.86:41588) 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.304+0000 7f51afe00700 1 -- 172.21.15.86:0/3761851905 learned_addr learned my addr 172.21.15.86:0/3761851905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51b4e00700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b0100d50 0x7f51b0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51b5800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51b01028e0 0x7f51b00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51afe00700 1 -- 172.21.15.86:0/3761851905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b0100d50 msgr2=0x7f51b0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51afe00700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b0100d50 0x7f51b0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51afe00700 1 -- 172.21.15.86:0/3761851905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51b01028e0 msgr2=0x7f51b00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51afe00700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51b01028e0 0x7f51b00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51afe00700 1 -- 172.21.15.86:0/3761851905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0095f0 con 0x7f51b0101f40 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51afe00700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b00731a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f519c00b580 tx=0x7f519c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c01d070 con 0x7f51b0101f40 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519c003f90 con 0x7f51b0101f40 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b01a0f60 con 0x7f51b0101f40 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.305+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c022c40 con 0x7f51b0101f40 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.306+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b01a1320 con 0x7f51b0101f40 2024-03-22T19:10:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.306+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b01081d0 con 0x7f51b0101f40 2024-03-22T19:10:12.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.308+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f519c022470 con 0x7f51b0101f40 2024-03-22T19:10:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.309+0000 7f51ad600700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51980a0b20 0x7f51980a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.310+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f519c0cd6c0 con 0x7f51b0101f40 2024-03-22T19:10:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.310+0000 7f51b4e00700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51980a0b20 0x7f51980a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.310+0000 7f51b4e00700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51980a0b20 0x7f51980a2fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f51b0101da0 tx=0x7f51a4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.313+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f519c0970d0 con 0x7f51b0101f40 2024-03-22T19:10:12.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2976037238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:12.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2424256584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.549+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51b002cdc0 con 0x7f51b0101f40 2024-03-22T19:10:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.549+0000 7f51ad600700 1 -- 172.21.15.86:0/3761851905 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f519c027020 con 0x7f51b0101f40 2024-03-22T19:10:12.550 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51980a0b20 msgr2=0x7f51980a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51980a0b20 0x7f51980a2fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f51b0101da0 tx=0x7f51a4007400 comp rx=0 tx=0).stop 2024-03-22T19:10:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 msgr2=0x7f51b00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b00731a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f519c00b580 tx=0x7f519c009580 comp rx=0 tx=0).stop 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 shutdown_connections 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f51980a0b20 0x7f51980a2fe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51b01028e0 0x7f51b00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b0100d50 0x7f51b0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 --2- 172.21.15.86:0/3761851905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51b0101f40 0x7f51b00731a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 >> 172.21.15.86:0/3761851905 conn(0x7f51b00fc2c0 msgr2=0x7f51b0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 shutdown_connections 2024-03-22T19:10:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.552+0000 7f51b7800700 1 -- 172.21.15.86:0/3761851905 wait complete. 2024-03-22T19:10:12.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:12.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2976037238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:12.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2424256584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:12.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2976037238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:12.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2424256584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 -- 172.21.15.86:0/574718921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8301027e0 msgr2=0x7fa830109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 --2- 172.21.15.86:0/574718921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8301027e0 0x7fa830109090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa82800b0d0 tx=0x7fa82800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 -- 172.21.15.86:0/574718921 shutdown_connections 2024-03-22T19:10:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 --2- 172.21.15.86:0/574718921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8301027e0 0x7fa830109090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 --2- 172.21.15.86:0/574718921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830100c50 0x7fa830101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 --2- 172.21.15.86:0/574718921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa830101e40 0x7fa8301022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.777+0000 7fa838600700 1 -- 172.21.15.86:0/574718921 >> 172.21.15.86:0/574718921 conn(0x7fa8300fc1c0 msgr2=0x7fa8300fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.778+0000 7fa838600700 1 -- 172.21.15.86:0/574718921 shutdown_connections 2024-03-22T19:10:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.778+0000 7fa838600700 1 -- 172.21.15.86:0/574718921 wait complete. 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 Processor -- start 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 -- start start 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 0x7fa830198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830101e40 0x7fa830199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8301027e0 0x7fa8301a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301a09e0 con 0x7fa830101e40 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301a0b20 con 0x7fa830100c50 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa838600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301a0cb0 con 0x7fa8301027e0 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa835c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 0x7fa830198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa835c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 0x7fa830198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43012/0 (socket says 172.21.15.86:43012) 2024-03-22T19:10:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa835c00700 1 -- 172.21.15.86:0/3593468041 learned_addr learned my addr 172.21.15.86:0/3593468041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa835200700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830101e40 0x7fa830199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.779+0000 7fa836600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8301027e0 0x7fa8301a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa836600700 1 -- 172.21.15.86:0/3593468041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 msgr2=0x7fa830198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa836600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 0x7fa830198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa836600700 1 -- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830101e40 msgr2=0x7fa830199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa836600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830101e40 0x7fa830199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa836600700 1 -- 172.21.15.86:0/3593468041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009d00 con 0x7fa8301027e0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa835c00700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 0x7fa830198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa836600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8301027e0 0x7fa8301a0410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa828007b60 tx=0x7fa828008e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82800e040 con 0x7fa8301027e0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa828005070 con 0x7fa8301027e0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8301a0f30 con 0x7fa8301027e0 2024-03-22T19:10:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82801d6d0 con 0x7fa8301027e0 2024-03-22T19:10:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.780+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8301a12f0 con 0x7fa8301027e0 2024-03-22T19:10:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.781+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8300679e0 con 0x7fa8301027e0 2024-03-22T19:10:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.784+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fa8280051e0 con 0x7fa8301027e0 2024-03-22T19:10:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.786+0000 7fa82e000700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8180a0ba0 0x7fa8180a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.786+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8280cce90 con 0x7fa8301027e0 2024-03-22T19:10:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.786+0000 7fa835c00700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8180a0ba0 0x7fa8180a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.787+0000 7fa835c00700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8180a0ba0 0x7fa8180a3060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa824005950 tx=0x7fa824009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:12.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:12.789+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa828096880 con 0x7fa8301027e0 2024-03-22T19:10:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.077+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8301a14e0 con 0x7fa8301027e0 2024-03-22T19:10:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.078+0000 7fa82e000700 1 -- 172.21.15.86:0/3593468041 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10924 (secure 0 0 0) 0x7fa828099ea0 con 0x7fa8301027e0 2024-03-22T19:10:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:13.079 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:09:21.111792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8180a0ba0 msgr2=0x7fa8180a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8180a0ba0 0x7fa8180a3060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa824005950 tx=0x7fa824009260 comp rx=0 tx=0).stop 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8301027e0 msgr2=0x7fa8301a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8301027e0 0x7fa8301a0410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa828007b60 tx=0x7fa828008e40 comp rx=0 tx=0).stop 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 shutdown_connections 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8180a0ba0 0x7fa8180a3060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8301027e0 0x7fa8301a0410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa830100c50 0x7fa830198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 --2- 172.21.15.86:0/3593468041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830101e40 0x7fa830199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 >> 172.21.15.86:0/3593468041 conn(0x7fa8300fc1c0 msgr2=0x7fa830072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 shutdown_connections 2024-03-22T19:10:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.081+0000 7fa838600700 1 -- 172.21.15.86:0/3593468041 wait complete. 2024-03-22T19:10:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:10:13.101 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 -- 172.21.15.86:0/1704061879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0100c50 msgr2=0x7fc1d0101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 --2- 172.21.15.86:0/1704061879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0100c50 0x7fc1d0101070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40099c0 tx=0x7fc1c4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 -- 172.21.15.86:0/1704061879 shutdown_connections 2024-03-22T19:10:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 --2- 172.21.15.86:0/1704061879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d01027e0 0x7fc1d0109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 --2- 172.21.15.86:0/1704061879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0100c50 0x7fc1d0101070 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 --2- 172.21.15.86:0/1704061879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1d0101e40 0x7fc1d01022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.308+0000 7fc1d8400700 1 -- 172.21.15.86:0/1704061879 >> 172.21.15.86:0/1704061879 conn(0x7fc1d00fc1c0 msgr2=0x7fc1d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.309+0000 7fc1d8400700 1 -- 172.21.15.86:0/1704061879 shutdown_connections 2024-03-22T19:10:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.309+0000 7fc1d8400700 1 -- 172.21.15.86:0/1704061879 wait complete. 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.310+0000 7fc1d8400700 1 Processor -- start 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.310+0000 7fc1d8400700 1 -- start start 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.310+0000 7fc1d8400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 0x7fc1d0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.310+0000 7fc1d8400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0101e40 0x7fc1d01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.310+0000 7fc1d8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1d01027e0 0x7fc1d01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.310+0000 7fc1d8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01a08f0 con 0x7fc1d01027e0 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d8400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01a0a60 con 0x7fc1d0101e40 2024-03-22T19:10:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d8400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01a0c20 con 0x7fc1d0100c50 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 0x7fc1d0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 0x7fc1d0198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36252/0 (socket says 172.21.15.86:36252) 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 -- 172.21.15.86:0/738824825 learned_addr learned my addr 172.21.15.86:0/738824825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d6400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1d01027e0 0x7fc1d01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5000700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0101e40 0x7fc1d01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 -- 172.21.15.86:0/738824825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0101e40 msgr2=0x7fc1d01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0101e40 0x7fc1d01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 -- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1d01027e0 msgr2=0x7fc1d01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1d01027e0 0x7fc1d01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 -- 172.21.15.86:0/738824825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40095f0 con 0x7fc1d0100c50 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.311+0000 7fc1d5a00700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 0x7fc1d0198e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006010 tx=0x7fc1c400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.312+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c401d070 con 0x7fc1d0100c50 2024-03-22T19:10:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.312+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c4005020 con 0x7fc1d0100c50 2024-03-22T19:10:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.312+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d01a0ea0 con 0x7fc1d0100c50 2024-03-22T19:10:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.312+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4004b10 con 0x7fc1d0100c50 2024-03-22T19:10:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.312+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d01a1210 con 0x7fc1d0100c50 2024-03-22T19:10:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.313+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d0067a50 con 0x7fc1d0100c50 2024-03-22T19:10:13.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.315+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc1c4005190 con 0x7fc1d0100c50 2024-03-22T19:10:13.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.316+0000 7fc1ce000700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1b80a0ba0 0x7fc1b80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.317+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc1c40cca60 con 0x7fc1d0100c50 2024-03-22T19:10:13.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.317+0000 7fc1d5000700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1b80a0ba0 0x7fc1b80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.317+0000 7fc1d5000700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1b80a0ba0 0x7fc1b80a3060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1d019a280 tx=0x7fc1bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:13.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.321+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1d0067a50 con 0x7fc1d0100c50 2024-03-22T19:10:13.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:13 smithi086 ceph-mon[38032]: pgmap v263: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:13.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:13 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:10:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3761851905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3593468041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:13 smithi167 ceph-mon[47728]: pgmap v263: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:13 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:10:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:13 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3761851905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:13 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3593468041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:13.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.611+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1d01a1460 con 0x7fc1d0100c50 2024-03-22T19:10:13.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.613+0000 7fc1ce000700 1 -- 172.21.15.86:0/738824825 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7fc1c409ee60 con 0x7fc1d0100c50 2024-03-22T19:10:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:13.614 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:13.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1b80a0ba0 msgr2=0x7fc1b80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1b80a0ba0 0x7fc1b80a3060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1d019a280 tx=0x7fc1bc009260 comp rx=0 tx=0).stop 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 msgr2=0x7fc1d0198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 0x7fc1d0198e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006010 tx=0x7fc1c400f690 comp rx=0 tx=0).stop 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 shutdown_connections 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1b80a0ba0 0x7fc1b80a3060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1d01027e0 0x7fc1d01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0100c50 0x7fc1d0198e90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 --2- 172.21.15.86:0/738824825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d0101e40 0x7fc1d01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 >> 172.21.15.86:0/738824825 conn(0x7fc1d00fc1c0 msgr2=0x7fc1d0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 shutdown_connections 2024-03-22T19:10:13.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.616+0000 7fc1d8400700 1 -- 172.21.15.86:0/738824825 wait complete. 2024-03-22T19:10:13.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:13 smithi131 ceph-mon[78559]: pgmap v263: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:13 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:10:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:13 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3761851905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:13 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3593468041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 -- 172.21.15.86:0/3739735513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 msgr2=0x7f5320073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 --2- 172.21.15.86:0/3739735513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320073c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f530c009990 tx=0x7f530c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 -- 172.21.15.86:0/3739735513 shutdown_connections 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 --2- 172.21.15.86:0/3739735513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5320108b10 0x7f532010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 --2- 172.21.15.86:0/3739735513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5320074df0 0x7f5320073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 --2- 172.21.15.86:0/3739735513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320073c90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 -- 172.21.15.86:0/3739735513 >> 172.21.15.86:0/3739735513 conn(0x7f53200fc1c0 msgr2=0x7f53200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 -- 172.21.15.86:0/3739735513 shutdown_connections 2024-03-22T19:10:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.844+0000 7f5326000700 1 -- 172.21.15.86:0/3739735513 wait complete. 2024-03-22T19:10:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.845+0000 7f5326000700 1 Processor -- start 2024-03-22T19:10:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 -- start start 2024-03-22T19:10:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5320074df0 0x7f5320199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5320108b10 0x7f53201a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201a0930 con 0x7f5320108b10 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201a0a70 con 0x7f5320074df0 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f5326000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201a0c00 con 0x7f5320073810 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f531f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f531f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36272/0 (socket says 172.21.15.86:36272) 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f531f400700 1 -- 172.21.15.86:0/3534969316 learned_addr learned my addr 172.21.15.86:0/3534969316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f531fe00700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5320108b10 0x7f53201a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.846+0000 7f531ea00700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5320074df0 0x7f5320199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f531f400700 1 -- 172.21.15.86:0/3534969316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5320074df0 msgr2=0x7f5320199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f531f400700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5320074df0 0x7f5320199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f531f400700 1 -- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5320108b10 msgr2=0x7f53201a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f531f400700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5320108b10 0x7f53201a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f531f400700 1 -- 172.21.15.86:0/3534969316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0095f0 con 0x7f5320073810 2024-03-22T19:10:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f531f400700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320198e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f531000e9a0 tx=0x7f531000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310009790 con 0x7f5320073810 2024-03-22T19:10:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5310010430 con 0x7f5320073810 2024-03-22T19:10:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53201a0ee0 con 0x7f5320073810 2024-03-22T19:10:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310018680 con 0x7f5320073810 2024-03-22T19:10:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.847+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53201a1310 con 0x7f5320073810 2024-03-22T19:10:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.848+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53201078d0 con 0x7f5320073810 2024-03-22T19:10:13.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.850+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f53100105a0 con 0x7f5320073810 2024-03-22T19:10:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.851+0000 7f5317e00700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f53040a0ba0 0x7f53040a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.851+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5310014070 con 0x7f5320073810 2024-03-22T19:10:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.852+0000 7f531ea00700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f53040a0ba0 0x7f53040a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.852+0000 7f531ea00700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f53040a0ba0 0x7f53040a3060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b680 tx=0x7f530c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:13.855+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f53100529f0 con 0x7f5320073810 2024-03-22T19:10:14.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.131+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53200679e0 con 0x7f5320073810 2024-03-22T19:10:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.132+0000 7f5317e00700 1 -- 172.21.15.86:0/3534969316 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f5310095b70 con 0x7f5320073810 2024-03-22T19:10:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:14.133 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:09:21.111792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:14.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f53040a0ba0 msgr2=0x7f53040a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f53040a0ba0 0x7f53040a3060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b680 tx=0x7f530c005fd0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 msgr2=0x7f5320198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320198e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f531000e9a0 tx=0x7f531000ecb0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 shutdown_connections 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f53040a0ba0 0x7f53040a3060 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5320108b10 0x7f53201a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5320073810 0x7f5320198e00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 --2- 172.21.15.86:0/3534969316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5320074df0 0x7f5320199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 >> 172.21.15.86:0/3534969316 conn(0x7f53200fc1c0 msgr2=0x7f5320104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 shutdown_connections 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.135+0000 7f5326000700 1 -- 172.21.15.86:0/3534969316 wait complete. 2024-03-22T19:10:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:10:14.156 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-22T19:09:13.618776+0000', 'modified': '2024-03-22T19:09:21.111792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24557, 'mds_2': 24550, 'mds_3': 24524, 'mds_4': 24523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14514': {'gid': 14514, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3022543840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3022543840}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3022543840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24470': {'gid': 24470, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/2878260168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 2878260168}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 2878260168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/700195904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 700195904}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 700195904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/4106390239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 4106390239}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 4106390239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/1695283976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 1695283976}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 1695283976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6835/2161889252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2161889252}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2161889252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.131:6837/3746834114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 3746834114}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 3746834114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.167:6839/3747705607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3747705607}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3747705607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6841/3747210669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3747210669}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3747210669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6839/1545428523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 1545428523}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 1545428523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-22T19:10:14.156 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-22T19:10:14.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 -- 172.21.15.86:0/3934739203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 msgr2=0x7faef41003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 --2- 172.21.15.86:0/3934739203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 0x7faef41003a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faee40098f0 tx=0x7faee4009c00 comp rx=0 tx=0).stop 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 -- 172.21.15.86:0/3934739203 shutdown_connections 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 --2- 172.21.15.86:0/3934739203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef41008e0 0x7faef4100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 --2- 172.21.15.86:0/3934739203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef40ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 --2- 172.21.15.86:0/3934739203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 0x7faef41003a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 -- 172.21.15.86:0/3934739203 >> 172.21.15.86:0/3934739203 conn(0x7faef40fb1c0 msgr2=0x7faef40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 -- 172.21.15.86:0/3934739203 shutdown_connections 2024-03-22T19:10:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.355+0000 7faefc800700 1 -- 172.21.15.86:0/3934739203 wait complete. 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.356+0000 7faefc800700 1 Processor -- start 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.356+0000 7faefc800700 1 -- start start 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefc800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefc800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 0x7faef4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefc800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef41008e0 0x7faef41a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefc800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41a0a50 con 0x7faef40ff5c0 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefc800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41a0b90 con 0x7faef41008e0 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefc800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41a0d20 con 0x7faef40fff20 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef4198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41664/0 (socket says 172.21.15.86:41664) 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 -- 172.21.15.86:0/1278600402 learned_addr learned my addr 172.21.15.86:0/1278600402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9400700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 0x7faef4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faefa800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef41008e0 0x7faef41a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 -- 172.21.15.86:0/1278600402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 msgr2=0x7faef4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 0x7faef4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 -- 172.21.15.86:0/1278600402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef41008e0 msgr2=0x7faef41a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef41008e0 0x7faef41a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.357+0000 7faef9e00700 1 -- 172.21.15.86:0/1278600402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80095f0 con 0x7faef40ff5c0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.358+0000 7faef9e00700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef4198f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faee800ec10 tx=0x7faee800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.358+0000 7faeeea00700 1 -- 172.21.15.86:0/1278600402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800cc90 con 0x7faef40ff5c0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.358+0000 7faeeea00700 1 -- 172.21.15.86:0/1278600402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee80044e0 con 0x7faef40ff5c0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.358+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee4009520 con 0x7faef40ff5c0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.358+0000 7faeeea00700 1 -- 172.21.15.86:0/1278600402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8010430 con 0x7faef40ff5c0 2024-03-22T19:10:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.358+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef41a1290 con 0x7faef40ff5c0 2024-03-22T19:10:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.361+0000 7faeeea00700 1 -- 172.21.15.86:0/1278600402 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7faee8003fb0 con 0x7faef40ff5c0 2024-03-22T19:10:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 Processor -- start 2024-03-22T19:10:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 -- start start 2024-03-22T19:10:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faeeea00700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faedc0a0b20 0x7faedc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faeeea00700 1 -- 172.21.15.86:0/1278600402 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7faee8014070 con 0x7faef40ff5c0 2024-03-22T19:10:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0813ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b0814df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b0815850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b076c940 con 0x5605b0813bd0 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b076a300 con 0x5605b08153d0 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.363+0000 7faf10258b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0765bf0 con 0x5605b0814530 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faefa800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faedc0a0b20 0x7faedc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0813ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0813ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41666/0 (socket says 172.21.15.86:41666) 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 -- 172.21.15.86:0/775799063 learned_addr learned my addr 172.21.15.86:0/775799063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9400700 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b0814df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faefa800700 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b0815850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 -- 172.21.15.86:0/775799063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 msgr2=0x5605b0814df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b0814df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 -- 172.21.15.86:0/775799063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 msgr2=0x5605b0815850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b0815850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faef9e00700 1 -- 172.21.15.86:0/775799063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605b05e7360 con 0x5605b0813bd0 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faefa800700 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b0815850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.364+0000 7faefa800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faedc0a0b20 0x7faedc0a2fe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faef000f540 tx=0x7faef0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.365+0000 7faef9e00700 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0813ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faee8005d70 tx=0x7faee80997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbac18f1d0a7f4f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.365+0000 7faee3000700 1 -- 172.21.15.86:0/775799063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80b1070 con 0x5605b0813bd0 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.365+0000 7faee3000700 1 -- 172.21.15.86:0/775799063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee809a320 con 0x5605b0813bd0 2024-03-22T19:10:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.365+0000 7faee3000700 1 -- 172.21.15.86:0/775799063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8004260 con 0x5605b0813bd0 2024-03-22T19:10:14.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.365+0000 7faf10258b80 1 -- 172.21.15.86:0/775799063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 msgr2=0x5605b0813ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.365+0000 7faf10258b80 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0813ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faee8005d70 tx=0x7faee80997e0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 -- 172.21.15.86:0/775799063 shutdown_connections 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b0815850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0813ff0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 --2- 172.21.15.86:0/775799063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b0814df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 -- 172.21.15.86:0/775799063 >> 172.21.15.86:0/775799063 conn(0x5605b07891d0 msgr2=0x5605b080eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 -- 172.21.15.86:0/775799063 shutdown_connections 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 -- 172.21.15.86:0/775799063 wait complete. 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.366+0000 7faf10258b80 1 -- start start 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 Processor -- start 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0813bd0 0x5605b081da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b081df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0824f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0701d30 con 0x5605b08153d0 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b07019d0 con 0x5605b0813bd0 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b05e9650 con 0x5605b0814530 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.367+0000 7faf10258b80 10 client.0 _ll_register_callbacks cb 0x5605b056d350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b081df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b081df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36312/0 (socket says 172.21.15.86:36312) 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 -- 172.21.15.86:0/812275760 learned_addr learned my addr 172.21.15.86:0/812275760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faefa800700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0824f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9e00700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0813bd0 0x5605b081da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 -- 172.21.15.86:0/812275760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0813bd0 msgr2=0x5605b081da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0813bd0 0x5605b081da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 -- 172.21.15.86:0/812275760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 msgr2=0x5605b0824f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0824f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 -- 172.21.15.86:0/812275760 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef41a1290 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faefa800700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0824f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faef9400700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b081df50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faee4000c00 tx=0x7faee400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b8b45ef3b0c23a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faed7400700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.368+0000 7faed7400700 1 -- 172.21.15.86:0/812275760 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4005240 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faed7400700 1 -- 172.21.15.86:0/812275760 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7faee400bbc0 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605b0806a80 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faed7400700 1 -- 172.21.15.86:0/812275760 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faee40053a0 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 10 client.24976 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5605b0593b10 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faed7400700 1 -- 172.21.15.86:0/812275760 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee4031020 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faed7400700 1 -- 172.21.15.86:0/812275760 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faee4015020 con 0x5605b0814530 2024-03-22T19:10:14.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 10 client.24976 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 10 client.24976 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 20 client.24976 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b08280a0 0x5605b082dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 4 client.24976 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.369+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605b075e6b0 con 0x5605b08280a0 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.370+0000 7faef9e00700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b08280a0 0x5605b082dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.370+0000 7faed7400700 10 client.24976 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.370+0000 7faef9e00700 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b08280a0 0x5605b082dec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a58837ae8e09e2 server_cookie=862089126b8e6bf9 in_seq=0 out_seq=0 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faed7400700 1 -- 172.21.15.86:0/812275760 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5605b075e6b0 con 0x5605b08280a0 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faed7400700 10 client.24976 handle_command_reply: tid=0 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 client.24976 shutdown 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 msgr2=0x5605b081df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b081df50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faee4000c00 tx=0x7faee400b790 comp rx=0 tx=0).stop 2024-03-22T19:10:14.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 shutdown_connections 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0824f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0813bd0 0x5605b081da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b08280a0 0x5605b082dec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 --2- 172.21.15.86:0/812275760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b081df50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 >> 172.21.15.86:0/812275760 conn(0x5605b07891d0 msgr2=0x5605b080e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faed7400700 0 client.24976 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 shutdown_connections 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 1 -- 172.21.15.86:0/812275760 wait complete. 2024-03-22T19:10:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.371+0000 7faf10258b80 20 client.24976 trim_cache size 0 max 16384 2024-03-22T19:10:14.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 Processor -- start 2024-03-22T19:10:14.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 -- start start 2024-03-22T19:10:14.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0822ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b08157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b081cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0762850 con 0x5605b0813bd0 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0704de0 con 0x5605b08153d0 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.389+0000 7faf10258b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b07d98f0 con 0x5605b0814530 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0822ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0822ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41682/0 (socket says 172.21.15.86:41682) 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 -- 172.21.15.86:0/3270931774 learned_addr learned my addr 172.21.15.86:0/3270931774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9400700 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b08157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faefa800700 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b081cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 -- 172.21.15.86:0/3270931774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 msgr2=0x5605b08157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b08157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 -- 172.21.15.86:0/3270931774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 msgr2=0x5605b081cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b081cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 -- 172.21.15.86:0/3270931774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605b072be50 con 0x5605b0813bd0 2024-03-22T19:10:14.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9400700 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b08157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:14.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.390+0000 7faef9e00700 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0822ea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faee80998d0 tx=0x7faee809a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fe681a98d2076e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faee3a00700 1 -- 172.21.15.86:0/3270931774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80b1040 con 0x5605b0813bd0 2024-03-22T19:10:14.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faee3a00700 1 -- 172.21.15.86:0/3270931774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee80a20a0 con 0x5605b0813bd0 2024-03-22T19:10:14.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faee3a00700 1 -- 172.21.15.86:0/3270931774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8099110 con 0x5605b0813bd0 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 -- 172.21.15.86:0/3270931774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 msgr2=0x5605b0822ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0822ea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faee80998d0 tx=0x7faee809a490 comp rx=0 tx=0).stop 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 -- 172.21.15.86:0/3270931774 shutdown_connections 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b08153d0 0x5605b081cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b0813bd0 0x5605b0822ea0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 --2- 172.21.15.86:0/3270931774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0814530 0x5605b08157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.394+0000 7faf10258b80 1 -- 172.21.15.86:0/3270931774 >> 172.21.15.86:0/3270931774 conn(0x5605b07891d0 msgr2=0x5605b081b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.395+0000 7faf10258b80 1 -- 172.21.15.86:0/3270931774 shutdown_connections 2024-03-22T19:10:14.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.395+0000 7faf10258b80 1 -- 172.21.15.86:0/3270931774 wait complete. 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.395+0000 7faf10258b80 1 -- start start 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 Processor -- start 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 0x5605b081df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0814530 0x5605b0821ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0822420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0822ae0 con 0x5605b08153d0 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0822c50 con 0x5605b0814530 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b0822e10 con 0x5605b0813bd0 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faf10258b80 10 client.0 _ll_register_callbacks cb 0x7faee400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:14.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faef9e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 0x5605b081df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faef9e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 0x5605b081df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36316/0 (socket says 172.21.15.86:36316) 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faef9e00700 1 -- 172.21.15.86:0/1368294495 learned_addr learned my addr 172.21.15.86:0/1368294495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.396+0000 7faefa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0822420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9400700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0814530 0x5605b0821ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9e00700 1 -- 172.21.15.86:0/1368294495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0814530 msgr2=0x5605b0821ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9e00700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0814530 0x5605b0821ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9e00700 1 -- 172.21.15.86:0/1368294495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 msgr2=0x5605b0822420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9e00700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0822420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9e00700 1 -- 172.21.15.86:0/1368294495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605b05e7360 con 0x5605b0813bd0 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9400700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0814530 0x5605b0821ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faef9e00700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 0x5605b081df20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faee8099920 tx=0x7faee8004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3de4df160283410 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faed7400700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:10:14.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faed7400700 1 -- 172.21.15.86:0/1368294495 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80b1040 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faed7400700 1 -- 172.21.15.86:0/1368294495 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7faee8099ef0 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faee809a320 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faed7400700 1 -- 172.21.15.86:0/1368294495 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faee809a0f0 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.397+0000 7faf10258b80 10 client.24982 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5605b05ebc80 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faed7400700 1 -- 172.21.15.86:0/1368294495 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee809a320 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faed7400700 1 -- 172.21.15.86:0/1368294495 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faee8095a10 con 0x5605b0813bd0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 10 client.24982 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 10 client.24982 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 20 client.24982 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b087f4a0 0x5605b087bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 4 client.24982 mds_command: new command op to 24523 tid=0[{"prefix": "status"}] 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5605b05b4ca0 con 0x5605b087f4a0 2024-03-22T19:10:14.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.398+0000 7faef9400700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b087f4a0 0x5605b087bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faed7400700 10 client.24982 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faef9400700 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b087f4a0 0x5605b087bed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad7a8f8e5ce629d4 server_cookie=f1c1da1314a48c9c in_seq=0 out_seq=0 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faed7400700 1 -- 172.21.15.86:0/1368294495 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5605b05b4ca0 con 0x5605b087f4a0 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faed7400700 10 client.24982 handle_command_reply: tid=0 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 client.24982 shutdown 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 msgr2=0x5605b081df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 0x5605b081df20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faee8099920 tx=0x7faee8004260 comp rx=0 tx=0).stop 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 shutdown_connections 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605b08153d0 0x5605b0822420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5605b087f4a0 0x5605b087bed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605b0813bd0 0x5605b081df20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 --2- 172.21.15.86:0/1368294495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605b0814530 0x5605b0821ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 >> 172.21.15.86:0/1368294495 conn(0x5605b07891d0 msgr2=0x5605b080ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faed7400700 0 client.24982 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:14.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 shutdown_connections 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 1 -- 172.21.15.86:0/1368294495 wait complete. 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.399+0000 7faf10258b80 20 client.24982 trim_cache size 0 max 16384 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24523, 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 54.241931940000001, 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 79.224075880000001 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faedc0a0b20 msgr2=0x7faedc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faedc0a0b20 0x7faedc0a2fe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faef000f540 tx=0x7faef0009920 comp rx=0 tx=0).stop 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 msgr2=0x7faef4198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef4198f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faee800ec10 tx=0x7faee800ef20 comp rx=0 tx=0).stop 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 shutdown_connections 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faedc0a0b20 0x7faedc0a2fe0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef41008e0 0x7faef41a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef40ff5c0 0x7faef4198f20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 --2- 172.21.15.86:0/1278600402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faef40fff20 0x7faef4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 >> 172.21.15.86:0/1278600402 conn(0x7faef40fb1c0 msgr2=0x7faef40fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 shutdown_connections 2024-03-22T19:10:14.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.400+0000 7faefc800700 1 -- 172.21.15.86:0/1278600402 wait complete. 2024-03-22T19:10:14.417 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/738824825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3534969316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:14.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 -- 172.21.15.86:0/1777540424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c100d50 msgr2=0x7f6d2c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 --2- 172.21.15.86:0/1777540424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c100d50 0x7f6d2c101170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6d200099c0 tx=0x7f6d20009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 -- 172.21.15.86:0/1777540424 shutdown_connections 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 --2- 172.21.15.86:0/1777540424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 --2- 172.21.15.86:0/1777540424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c100d50 0x7f6d2c101170 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 --2- 172.21.15.86:0/1777540424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 0x7f6d2c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 -- 172.21.15.86:0/1777540424 >> 172.21.15.86:0/1777540424 conn(0x7f6d2c0fc2c0 msgr2=0x7f6d2c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 -- 172.21.15.86:0/1777540424 shutdown_connections 2024-03-22T19:10:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.621+0000 7f6d33c00700 1 -- 172.21.15.86:0/1777540424 wait complete. 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.622+0000 7f6d33c00700 1 Processor -- start 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 -- start start 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d2c100d50 0x7f6d2c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 0x7f6d2c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c073e80 con 0x7f6d2c1028e0 2024-03-22T19:10:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c073ff0 con 0x7f6d2c101f40 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d33c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c1a0d90 con 0x7f6d2c100d50 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d31c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d31c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41718/0 (socket says 172.21.15.86:41718) 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d31c00700 1 -- 172.21.15.86:0/2079508892 learned_addr learned my addr 172.21.15.86:0/2079508892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 0x7f6d2c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d31200700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d2c100d50 0x7f6d2c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d31c00700 1 -- 172.21.15.86:0/2079508892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d2c100d50 msgr2=0x7f6d2c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.623+0000 7f6d31c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d2c100d50 0x7f6d2c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d31c00700 1 -- 172.21.15.86:0/2079508892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 msgr2=0x7f6d2c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d31c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 0x7f6d2c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d31c00700 1 -- 172.21.15.86:0/2079508892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d200095f0 con 0x7f6d2c1028e0 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d2be00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 0x7f6d2c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d31c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c0736e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400ef30 tx=0x7f6d2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d29600700 1 -- 172.21.15.86:0/2079508892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24018070 con 0x7f6d2c1028e0 2024-03-22T19:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2c1a0fc0 con 0x7f6d2c1028e0 2024-03-22T19:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d29600700 1 -- 172.21.15.86:0/2079508892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2400f040 con 0x7f6d2c1028e0 2024-03-22T19:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d29600700 1 -- 172.21.15.86:0/2079508892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400cca0 con 0x7f6d2c1028e0 2024-03-22T19:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.624+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c1a1490 con 0x7f6d2c1028e0 2024-03-22T19:10:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.628+0000 7f6d477b6b80 1 Processor -- start 2024-03-22T19:10:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d29600700 1 -- 172.21.15.86:0/2079508892 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f6d24003a00 con 0x7f6d2c1028e0 2024-03-22T19:10:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 -- start start 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474526ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474527df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474528850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b47447f940 con 0x55b474527530 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b47447d300 con 0x55b474526bd0 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b474478bf0 con 0x55b4745283d0 2024-03-22T19:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.629+0000 7f6d31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474527df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474527df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41734/0 (socket says 172.21.15.86:41734) 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 -- 172.21.15.86:0/86159062 learned_addr learned my addr 172.21.15.86:0/86159062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d2be00700 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474528850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d2be00700 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474526ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 -- 172.21.15.86:0/86159062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 msgr2=0x55b474528850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474528850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 -- 172.21.15.86:0/86159062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 msgr2=0x55b474526ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474526ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 -- 172.21.15.86:0/86159062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4742fa360 con 0x55b474527530 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d2be00700 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474526ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d31200700 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474527df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006010 tx=0x7f6d20005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7a63dd3c7e571d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d1e000700 1 -- 172.21.15.86:0/86159062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000ba10 con 0x55b474527530 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d1e000700 1 -- 172.21.15.86:0/86159062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2001e040 con 0x55b474527530 2024-03-22T19:10:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.630+0000 7f6d1e000700 1 -- 172.21.15.86:0/86159062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20003ad0 con 0x55b474527530 2024-03-22T19:10:14.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.631+0000 7f6d29600700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6d140a0b20 0x7f6d140a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.631+0000 7f6d29600700 1 -- 172.21.15.86:0/2079508892 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6d240cbd00 con 0x7f6d2c1028e0 2024-03-22T19:10:14.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.631+0000 7f6d2be00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6d140a0b20 0x7f6d140a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.631+0000 7f6d477b6b80 1 -- 172.21.15.86:0/86159062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 msgr2=0x55b474527df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.631+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474527df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006010 tx=0x7f6d20005bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.631+0000 7f6d477b6b80 1 -- 172.21.15.86:0/86159062 shutdown_connections 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474528850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474526ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/86159062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474527df0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 -- 172.21.15.86:0/86159062 >> 172.21.15.86:0/86159062 conn(0x55b47449c1d0 msgr2=0x55b474521b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 -- 172.21.15.86:0/86159062 shutdown_connections 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d2be00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6d140a0b20 0x7f6d140a2fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55b474527bb0 tx=0x7f6d18013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 -- 172.21.15.86:0/86159062 wait complete. 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 -- start start 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.632+0000 7f6d477b6b80 1 Processor -- start 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474530a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474537f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b474414d30 con 0x55b474527530 2024-03-22T19:10:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4744149d0 con 0x55b474526bd0 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4742fc650 con 0x55b4745283d0 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d477b6b80 10 client.0 _ll_register_callbacks cb 0x55b474280350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41742/0 (socket says 172.21.15.86:41742) 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 -- 172.21.15.86:0/2773091174 learned_addr learned my addr 172.21.15.86:0/2773091174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d31c00700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474537f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d31200700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474530a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 -- 172.21.15.86:0/2773091174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 msgr2=0x55b474537f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474537f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 -- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 msgr2=0x55b474530a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.633+0000 7f6d2be00700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474530a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d2be00700 1 -- 172.21.15.86:0/2773091174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c109ae0 con 0x55b474527530 2024-03-22T19:10:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d2be00700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800f820 tx=0x7f6d180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b6b78064bc214 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d12a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d12a00700 1 -- 172.21.15.86:0/2773091174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18016040 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d12a00700 1 -- 172.21.15.86:0/2773091174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6d18003910 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b474519a80 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d12a00700 1 -- 172.21.15.86:0/2773091174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d18003ad0 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d477b6b80 10 client.14961 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.634+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b4742a6b10 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d12a00700 1 -- 172.21.15.86:0/2773091174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b474519a80 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d12a00700 1 -- 172.21.15.86:0/2773091174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d1802b440 con 0x55b474527530 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d477b6b80 10 client.14961 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d477b6b80 10 client.14961 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d477b6b80 20 client.14961 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b47453b140 0x55b474540f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d477b6b80 4 client.14961 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4744716b0 con 0x55b47453b140 2024-03-22T19:10:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.635+0000 7f6d31200700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b47453b140 0x55b474540f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.636+0000 7f6d31200700 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b47453b140 0x55b474540f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fb12646afcc7c38 server_cookie=2122ecf9f35cd1e8 in_seq=0 out_seq=0 2024-03-22T19:10:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.636+0000 7f6d12a00700 10 client.14961 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d12a00700 1 -- 172.21.15.86:0/2773091174 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b4744716b0 con 0x55b47453b140 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d12a00700 10 client.14961 handle_command_reply: tid=0 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 client.14961 shutdown 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 msgr2=0x55b474530f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800f820 tx=0x7f6d180047c0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 shutdown_connections 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b47453b140 0x55b474540f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b474537f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474530a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d12a00700 0 client.14961 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/2773091174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530f50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 >> 172.21.15.86:0/2773091174 conn(0x55b47449c1d0 msgr2=0x55b4745211a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 shutdown_connections 2024-03-22T19:10:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 1 -- 172.21.15.86:0/2773091174 wait complete. 2024-03-22T19:10:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.637+0000 7f6d477b6b80 20 client.14961 trim_cache size 0 max 16384 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.655+0000 7f6d477b6b80 1 Processor -- start 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.655+0000 7f6d477b6b80 1 -- start start 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.655+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.655+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474538640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.655+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4744ec8f0 con 0x55b474527530 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b474539d20 con 0x55b474526bd0 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b47451c1c0 con 0x55b4745283d0 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474538640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474538640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41754/0 (socket says 172.21.15.86:41754) 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 -- 172.21.15.86:0/3591436876 learned_addr learned my addr 172.21.15.86:0/3591436876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d31200700 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d31c00700 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 -- 172.21.15.86:0/3591436876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 msgr2=0x55b47452ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 -- 172.21.15.86:0/3591436876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 msgr2=0x55b474535f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 -- 172.21.15.86:0/3591436876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b47443ee50 con 0x55b474527530 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d31200700 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.656+0000 7f6d2be00700 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474538640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d18013010 tx=0x7f6d180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ea7a11f43fdada server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.657+0000 7f6d1ea00700 1 -- 172.21.15.86:0/3591436876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18016070 con 0x55b474527530 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.657+0000 7f6d1ea00700 1 -- 172.21.15.86:0/3591436876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d1802b540 con 0x55b474527530 2024-03-22T19:10:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.657+0000 7f6d1ea00700 1 -- 172.21.15.86:0/3591436876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18021640 con 0x55b474527530 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.657+0000 7f6d477b6b80 1 -- 172.21.15.86:0/3591436876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 msgr2=0x55b474538640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.657+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474538640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d18013010 tx=0x7f6d180118d0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 -- 172.21.15.86:0/3591436876 shutdown_connections 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/3591436876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474538640 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 -- 172.21.15.86:0/3591436876 >> 172.21.15.86:0/3591436876 conn(0x55b47449c1d0 msgr2=0x55b47452e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 -- 172.21.15.86:0/3591436876 shutdown_connections 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 -- 172.21.15.86:0/3591436876 wait complete. 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 -- start start 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.658+0000 7f6d477b6b80 1 Processor -- start 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b47452da40 con 0x55b474527530 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b47452dbb0 con 0x55b474526bd0 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b47452dd70 con 0x55b4745283d0 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d477b6b80 10 client.0 _ll_register_callbacks cb 0x7f6d18004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36354/0 (socket says 172.21.15.86:36354) 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 -- 172.21.15.86:0/1692777650 learned_addr learned my addr 172.21.15.86:0/1692777650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d2be00700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31200700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 -- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 msgr2=0x55b474535ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 -- 172.21.15.86:0/1692777650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 msgr2=0x55b474530fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.659+0000 7f6d31c00700 1 -- 172.21.15.86:0/1692777650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4742fa360 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d31c00700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d240957b0 tx=0x7f6d24098c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=839bc2c8bfb539ff server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d12a00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d12a00700 1 -- 172.21.15.86:0/1692777650 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240999b0 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d12a00700 1 -- 172.21.15.86:0/1692777650 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6d240af070 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d2001e040 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d12a00700 1 -- 172.21.15.86:0/1692777650 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d240992b0 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d477b6b80 10 client.25000 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b4742a6b10 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d12a00700 1 -- 172.21.15.86:0/1692777650 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d24063080 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.660+0000 7f6d12a00700 1 -- 172.21.15.86:0/1692777650 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d240987a0 con 0x55b4745283d0 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d477b6b80 10 client.25000 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d477b6b80 10 client.25000 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-22T19:10:14.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d477b6b80 20 client.25000 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:14.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b474592be0 0x55b47458f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d477b6b80 4 client.25000 mds_command: new command op to 24524 tid=0[{"prefix": "status"}] 2024-03-22T19:10:14.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b4742c7ca0 con 0x55b474592be0 2024-03-22T19:10:14.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.661+0000 7f6d31200700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b474592be0 0x55b47458f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d31200700 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b474592be0 0x55b47458f660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9212e96fe7187bc server_cookie=bdee6cbae891ef11 in_seq=0 out_seq=0 2024-03-22T19:10:14.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d12a00700 10 client.25000 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:14.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d12a00700 1 -- 172.21.15.86:0/1692777650 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b4742c7ca0 con 0x55b474592be0 2024-03-22T19:10:14.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d12a00700 10 client.25000 handle_command_reply: tid=0 2024-03-22T19:10:14.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 client.25000 shutdown 2024-03-22T19:10:14.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 msgr2=0x55b47452d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d240957b0 tx=0x7f6d24098c10 comp rx=0 tx=0).stop 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 shutdown_connections 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55b474592be0 0x55b47458f660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4745283d0 0x55b47452d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b474526bd0 0x55b474535ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d12a00700 0 client.25000 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 --2- 172.21.15.86:0/1692777650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b474527530 0x55b474530fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 >> 172.21.15.86:0/1692777650 conn(0x55b47449c1d0 msgr2=0x55b474521a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 shutdown_connections 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.662+0000 7f6d477b6b80 1 -- 172.21.15.86:0/1692777650 wait complete. 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d477b6b80 20 client.25000 trim_cache size 0 max 16384 2024-03-22T19:10:14.667 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24524, 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 55.517494278000001, 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 75.979529743000001 2024-03-22T19:10:14.668 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6d140a0b20 msgr2=0x7f6d140a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6d140a0b20 0x7f6d140a2fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55b474527bb0 tx=0x7f6d18013040 comp rx=0 tx=0).stop 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 msgr2=0x7f6d2c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c0736e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400ef30 tx=0x7f6d2400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 shutdown_connections 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6d140a0b20 0x7f6d140a2fe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d2c1028e0 0x7f6d2c0736e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d2c100d50 0x7f6d2c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 --2- 172.21.15.86:0/2079508892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2c101f40 0x7f6d2c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 >> 172.21.15.86:0/2079508892 conn(0x7f6d2c0fc2c0 msgr2=0x7f6d2c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 shutdown_connections 2024-03-22T19:10:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.663+0000 7f6d33c00700 1 -- 172.21.15.86:0/2079508892 wait complete. 2024-03-22T19:10:14.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-22T19:10:14.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/738824825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3534969316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/738824825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3534969316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 -- 172.21.15.86:0/2428527075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 msgr2=0x7f8b48073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 --2- 172.21.15.86:0/2428527075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b48073c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b340098f0 tx=0x7f8b34009c00 comp rx=0 tx=0).stop 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 -- 172.21.15.86:0/2428527075 shutdown_connections 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 --2- 172.21.15.86:0/2428527075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4810d320 0x7f8b4810f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 --2- 172.21.15.86:0/2428527075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b48074df0 0x7f8b48073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 --2- 172.21.15.86:0/2428527075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b48073c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 -- 172.21.15.86:0/2428527075 >> 172.21.15.86:0/2428527075 conn(0x7f8b480fc2c0 msgr2=0x7f8b480fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 -- 172.21.15.86:0/2428527075 shutdown_connections 2024-03-22T19:10:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.880+0000 7f8b4e200700 1 -- 172.21.15.86:0/2428527075 wait complete. 2024-03-22T19:10:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 Processor -- start 2024-03-22T19:10:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 -- start start 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b4819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b48074df0 0x7f8b4819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4810d320 0x7f8b481a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a4e90 con 0x7f8b48074df0 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a4fd0 con 0x7f8b48073810 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b4e200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a5160 con 0x7f8b4810d320 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b47400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b4819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b48074df0 0x7f8b4819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b47400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b4819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43136/0 (socket says 172.21.15.86:43136) 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.881+0000 7f8b47400700 1 -- 172.21.15.86:0/3121423273 learned_addr learned my addr 172.21.15.86:0/3121423273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47e00700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4810d320 0x7f8b481a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47400700 1 -- 172.21.15.86:0/3121423273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4810d320 msgr2=0x7f8b481a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47400700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4810d320 0x7f8b481a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47400700 1 -- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b48074df0 msgr2=0x7f8b4819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47400700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b48074df0 0x7f8b4819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47400700 1 -- 172.21.15.86:0/3121423273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34009520 con 0x7f8b48073810 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b47400700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b4819d3b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800e9c0 tx=0x7f8b3800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b3fe00700 1 -- 172.21.15.86:0/3121423273 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38004a30 con 0x7f8b48073810 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b3fe00700 1 -- 172.21.15.86:0/3121423273 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3800cc80 con 0x7f8b48073810 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b481a5440 con 0x7f8b48073810 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.882+0000 7f8b3fe00700 1 -- 172.21.15.86:0/3121423273 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380054e0 con 0x7f8b48073810 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.883+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481a5950 con 0x7f8b48073810 2024-03-22T19:10:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.885+0000 7f8b3fe00700 1 -- 172.21.15.86:0/3121423273 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f8b38004b90 con 0x7f8b48073810 2024-03-22T19:10:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 Processor -- start 2024-03-22T19:10:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b3fe00700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b2c0a0b20 0x7f8b2c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 -- start start 2024-03-22T19:10:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b3fe00700 1 -- 172.21.15.86:0/3121423273 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b38014070 con 0x7f8b48073810 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b47e00700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b2c0a0b20 0x7f8b2c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f3dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f3edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f3f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4e96940 con 0x5572c4f3e530 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4e94300 con 0x5572c4f3f3d0 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.887+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4e8fbf0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f3edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b47400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f3dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b47400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f3dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36374/0 (socket says 172.21.15.86:36374) 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b47400700 1 -- 172.21.15.86:0/3461817224 learned_addr learned my addr 172.21.15.86:0/3461817224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b47e00700 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f3f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 -- 172.21.15.86:0/3461817224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 msgr2=0x5572c4f3dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f3dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 -- 172.21.15.86:0/3461817224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 msgr2=0x5572c4f3f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f3f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 -- 172.21.15.86:0/3461817224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c4d11360 con 0x5572c4f3e530 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b47400700 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f3dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b47e00700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b2c0a0b20 0x7f8b2c0a2fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000f870 tx=0x7f8b4000fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.888+0000 7f8b46a00700 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f3edf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009230 tx=0x7f8b34009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd3c5172914cd26 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b33e00700 1 -- 172.21.15.86:0/3461817224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400f660 con 0x5572c4f3e530 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b33e00700 1 -- 172.21.15.86:0/3461817224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3401d040 con 0x5572c4f3e530 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b33e00700 1 -- 172.21.15.86:0/3461817224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34005440 con 0x5572c4f3e530 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3461817224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 msgr2=0x5572c4f3edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f3edf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009230 tx=0x7f8b34009f50 comp rx=0 tx=0).stop 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3461817224 shutdown_connections 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f3f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f3dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3461817224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f3edf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.889+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3461817224 >> 172.21.15.86:0/3461817224 conn(0x5572c4eb31d0 msgr2=0x5572c4f38b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.890+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3461817224 shutdown_connections 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.890+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3461817224 wait complete. 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.890+0000 7f8b61c4db80 1 -- start start 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.890+0000 7f8b61c4db80 1 Processor -- start 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 0x5572c4f47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3e530 0x5572c4f47f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 0x5572c4f4ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4e2bd30 con 0x5572c4f3f3d0 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4e2b9d0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4d13650 con 0x5572c4f3e530 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b61c4db80 10 client.0 _ll_register_callbacks cb 0x5572c4c97350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 0x5572c4f4ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 0x5572c4f47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 0x5572c4f4ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41802/0 (socket says 172.21.15.86:41802) 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 0x5572c4f47a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43144/0 (socket says 172.21.15.86:43144) 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47e00700 1 -- 172.21.15.86:0/3503830588 learned_addr learned my addr 172.21.15.86:0/3503830588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b46a00700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3e530 0x5572c4f47f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 -- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3e530 msgr2=0x5572c4f47f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3e530 0x5572c4f47f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 -- 172.21.15.86:0/3503830588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 msgr2=0x5572c4f4ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 0x5572c4f4ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47400700 1 -- 172.21.15.86:0/3503830588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481a5950 con 0x5572c4f3dbd0 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.891+0000 7f8b47e00700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 0x5572c4f4ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b47400700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 0x5572c4f47a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b38096200 tx=0x7f8b3800cdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f8a497fe1c27e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b30c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b30c00700 1 -- 172.21.15.86:0/3503830588 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38099790 con 0x5572c4f3dbd0 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b30c00700 1 -- 172.21.15.86:0/3503830588 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b3809d5c0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572c4f30a80 con 0x5572c4f3dbd0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b30c00700 1 -- 172.21.15.86:0/3503830588 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b380a1c10 con 0x5572c4f3dbd0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b61c4db80 10 client.25007 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f8b481100c0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.892+0000 7f8b30c00700 1 -- 172.21.15.86:0/3503830588 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572c4f30a80 con 0x5572c4f3dbd0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b30c00700 1 -- 172.21.15.86:0/3503830588 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b380999f0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b61c4db80 10 client.25007 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b61c4db80 10 client.25007 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b61c4db80 20 client.25007 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4f52bd0 0x5572c4f57ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b61c4db80 4 client.25007 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572c4e886b0 con 0x5572c4f52bd0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.893+0000 7f8b46a00700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4f52bd0 0x5572c4f57ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.894+0000 7f8b46a00700 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4f52bd0 0x5572c4f57ef0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd10ca8afe83206d server_cookie=84c34d26d0f1545a in_seq=0 out_seq=0 2024-03-22T19:10:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.894+0000 7f8b30c00700 10 client.25007 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b30c00700 1 -- 172.21.15.86:0/3503830588 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5572c4e886b0 con 0x5572c4f52bd0 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b30c00700 10 client.25007 handle_command_reply: tid=0 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 client.25007 shutdown 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 msgr2=0x5572c4f47a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 0x5572c4f47a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b38096200 tx=0x7f8b3800cdf0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 shutdown_connections 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4f52bd0 0x5572c4f57ef0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3f3d0 0x5572c4f4ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b30c00700 0 client.25007 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3dbd0 0x5572c4f47a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/3503830588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3e530 0x5572c4f47f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 >> 172.21.15.86:0/3503830588 conn(0x5572c4eb31d0 msgr2=0x5572c4f381a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 shutdown_connections 2024-03-22T19:10:14.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 1 -- 172.21.15.86:0/3503830588 wait complete. 2024-03-22T19:10:14.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.895+0000 7f8b61c4db80 20 client.25007 trim_cache size 0 max 16384 2024-03-22T19:10:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.913+0000 7f8b61c4db80 1 Processor -- start 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 -- start start 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f46f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4f54470 con 0x5572c4f3e530 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4f331c0 con 0x5572c4f3f3d0 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4f355b0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f46f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f46f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41810/0 (socket says 172.21.15.86:41810) 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43152/0 (socket says 172.21.15.86:43152) 2024-03-22T19:10:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b46a00700 1 -- 172.21.15.86:0/109294418 learned_addr learned my addr 172.21.15.86:0/109294418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b47400700 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.914+0000 7f8b46a00700 1 -- 172.21.15.86:0/109294418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 msgr2=0x5572c4f4cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b46a00700 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b46a00700 1 -- 172.21.15.86:0/109294418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 msgr2=0x5572c4f466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b46a00700 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b46a00700 1 -- 172.21.15.86:0/109294418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c4cbdb10 con 0x5572c4f3e530 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b47e00700 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b46a00700 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f46f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009230 tx=0x7f8b34009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4180e8dfe6a38ae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b3d200700 1 -- 172.21.15.86:0/109294418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340052b0 con 0x5572c4f3e530 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b3d200700 1 -- 172.21.15.86:0/109294418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3400f580 con 0x5572c4f3e530 2024-03-22T19:10:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.915+0000 7f8b3d200700 1 -- 172.21.15.86:0/109294418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34003680 con 0x5572c4f3e530 2024-03-22T19:10:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 -- 172.21.15.86:0/109294418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 msgr2=0x5572c4f46f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f46f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009230 tx=0x7f8b34009260 comp rx=0 tx=0).stop 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 -- 172.21.15.86:0/109294418 shutdown_connections 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3f3d0 0x5572c4f466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/109294418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3e530 0x5572c4f46f20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 -- 172.21.15.86:0/109294418 >> 172.21.15.86:0/109294418 conn(0x5572c4eb31d0 msgr2=0x5572c4f458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 -- 172.21.15.86:0/109294418 shutdown_connections 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 -- 172.21.15.86:0/109294418 wait complete. 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 -- start start 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.916+0000 7f8b61c4db80 1 Processor -- start 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3e530 0x5572c4f4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3f3d0 0x5572c4f49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4f49d50 con 0x5572c4f3dbd0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4f49ec0 con 0x5572c4f3e530 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4f4a080 con 0x5572c4f3f3d0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b61c4db80 10 client.0 _ll_register_callbacks cb 0x7f8b380990f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41818/0 (socket says 172.21.15.86:41818) 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 -- 172.21.15.86:0/2515569474 learned_addr learned my addr 172.21.15.86:0/2515569474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b46a00700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3e530 0x5572c4f4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47e00700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3f3d0 0x5572c4f49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 -- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3f3d0 msgr2=0x5572c4f49670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3f3d0 0x5572c4f49670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 -- 172.21.15.86:0/2515569474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3e530 msgr2=0x5572c4f4ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3e530 0x5572c4f4ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.917+0000 7f8b47400700 1 -- 172.21.15.86:0/2515569474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b40009340 con 0x5572c4f3dbd0 2024-03-22T19:10:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b46a00700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3e530 0x5572c4f4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b47400700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4c990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b38095c90 tx=0x7f8b38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b41a198d6e691d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b30c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b30c00700 1 -- 172.21.15.86:0/2515569474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3809d5c0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b30c00700 1 -- 172.21.15.86:0/2515569474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b38099790 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572c4cdeca0 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b30c00700 1 -- 172.21.15.86:0/2515569474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b380a1c10 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b61c4db80 10 client.14985 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5572c4d11360 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.918+0000 7f8b30c00700 1 -- 172.21.15.86:0/2515569474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b380a5080 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b30c00700 1 -- 172.21.15.86:0/2515569474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b38081070 con 0x5572c4f3dbd0 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b61c4db80 10 client.14985 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b61c4db80 10 client.14985 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-22T19:10:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b61c4db80 20 client.14985 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4fa9970 0x5572c4fa6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b61c4db80 4 client.14985 mds_command: new command op to 24550 tid=0[{"prefix": "status"}] 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5572c4d13c90 con 0x5572c4fa9970 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.919+0000 7f8b46a00700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4fa9970 0x5572c4fa6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b46a00700 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4fa9970 0x5572c4fa6300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59654540a9ba3aa server_cookie=cebb9cf3c38f5d6e in_seq=0 out_seq=0 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b30c00700 10 client.14985 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b30c00700 1 -- 172.21.15.86:0/2515569474 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5572c4d13c90 con 0x5572c4fa9970 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b30c00700 10 client.14985 handle_command_reply: tid=0 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 client.14985 shutdown 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 msgr2=0x5572c4f4c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4c990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b38095c90 tx=0x7f8b38000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 shutdown_connections 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5572c4fa9970 0x5572c4fa6300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572c4f3f3d0 0x5572c4f49670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c4f3dbd0 0x5572c4f4c990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 --2- 172.21.15.86:0/2515569474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572c4f3e530 0x5572c4f4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 >> 172.21.15.86:0/2515569474 conn(0x5572c4eb31d0 msgr2=0x5572c4f38a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b30c00700 0 client.14985 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 shutdown_connections 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 1 -- 172.21.15.86:0/2515569474 wait complete. 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.920+0000 7f8b61c4db80 20 client.14985 trim_cache size 0 max 16384 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24550, 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 56.784028972999998, 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 74.328538593999994 2024-03-22T19:10:14.927 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b2c0a0b20 msgr2=0x7f8b2c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b2c0a0b20 0x7f8b2c0a2fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000f870 tx=0x7f8b4000fc30 comp rx=0 tx=0).stop 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 msgr2=0x7f8b4819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b4819d3b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800e9c0 tx=0x7f8b3800ecd0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 shutdown_connections 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b2c0a0b20 0x7f8b2c0a2fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4810d320 0x7f8b481a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b48073810 0x7f8b4819d3b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 --2- 172.21.15.86:0/3121423273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b48074df0 0x7f8b4819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 >> 172.21.15.86:0/3121423273 conn(0x7f8b480fc2c0 msgr2=0x7f8b481006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 shutdown_connections 2024-03-22T19:10:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:14.921+0000 7f8b4e200700 1 -- 172.21.15.86:0/3121423273 wait complete. 2024-03-22T19:10:14.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T19:10:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 -- 172.21.15.86:0/227845071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c4101f40 msgr2=0x7f79c41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 --2- 172.21.15.86:0/227845071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c4101f40 0x7f79c41023a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79b00099c0 tx=0x7f79b0009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 -- 172.21.15.86:0/227845071 shutdown_connections 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 --2- 172.21.15.86:0/227845071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c41028e0 0x7f79c4109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 --2- 172.21.15.86:0/227845071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4100d50 0x7f79c4101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 --2- 172.21.15.86:0/227845071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c4101f40 0x7f79c41023a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 -- 172.21.15.86:0/227845071 >> 172.21.15.86:0/227845071 conn(0x7f79c40fc2c0 msgr2=0x7f79c40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 -- 172.21.15.86:0/227845071 shutdown_connections 2024-03-22T19:10:15.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.135+0000 7f79cac00700 1 -- 172.21.15.86:0/227845071 wait complete. 2024-03-22T19:10:15.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 Processor -- start 2024-03-22T19:10:15.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 -- start start 2024-03-22T19:10:15.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c4100d50 0x7f79c4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4101f40 0x7f79c4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 0x7f79c41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41a0a30 con 0x7f79c41028e0 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41a0b70 con 0x7f79c4101f40 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79cac00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41a0d00 con 0x7f79c4100d50 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.136+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 0x7f79c41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 0x7f79c41a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41836/0 (socket says 172.21.15.86:41836) 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c4100d50 0x7f79c4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4101f40 0x7f79c4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c8c00700 1 -- 172.21.15.86:0/1610340635 learned_addr learned my addr 172.21.15.86:0/1610340635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 -- 172.21.15.86:0/1610340635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c4100d50 msgr2=0x7f79c4198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c4100d50 0x7f79c4198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 -- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 msgr2=0x7f79c41a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 0x7f79c41a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 -- 172.21.15.86:0/1610340635 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00095f0 con 0x7f79c4101f40 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3e00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c4100d50 0x7f79c4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c8c00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 0x7f79c41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c3400700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4101f40 0x7f79c4199440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009fd0 tx=0x7f79b000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.137+0000 7f79c0c00700 1 -- 172.21.15.86:0/1610340635 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b001d070 con 0x7f79c4101f40 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.138+0000 7f79c0c00700 1 -- 172.21.15.86:0/1610340635 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b000bda0 con 0x7f79c4101f40 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.138+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c41a0f80 con 0x7f79c4101f40 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.138+0000 7f79c0c00700 1 -- 172.21.15.86:0/1610340635 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0004b00 con 0x7f79c4101f40 2024-03-22T19:10:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.138+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c41a13a0 con 0x7f79c4101f40 2024-03-22T19:10:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.140+0000 7f79c0c00700 1 -- 172.21.15.86:0/1610340635 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f79b0004c60 con 0x7f79c4101f40 2024-03-22T19:10:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79c0c00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79a80a0ba0 0x7f79a80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 Processor -- start 2024-03-22T19:10:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79c0c00700 1 -- 172.21.15.86:0/1610340635 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f79b00ccee0 con 0x7f79c4101f40 2024-03-22T19:10:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79c3e00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79a80a0ba0 0x7f79a80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 -- start start 2024-03-22T19:10:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1477bd0 0x55c4c1477ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1478530 0x55c4c1478df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c14793d0 0x55c4c1479850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c13d0940 con 0x55c4c14793d0 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c13ce300 con 0x55c4c1477bd0 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.142+0000 7f79de733b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c13c9bf0 con 0x55c4c1478530 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3e00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79a80a0ba0 0x7f79a80a3060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79c4101da0 tx=0x7f79b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1477bd0 0x55c4c1477ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1478530 0x55c4c1478df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1477bd0 0x55c4c1477ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43178/0 (socket says 172.21.15.86:43178) 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c14793d0 0x55c4c1479850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3e00700 1 -- 172.21.15.86:0/3242711787 learned_addr learned my addr 172.21.15.86:0/3242711787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 -- 172.21.15.86:0/3242711787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1477bd0 msgr2=0x55c4c1477ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1477bd0 0x55c4c1477ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 -- 172.21.15.86:0/3242711787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c14793d0 msgr2=0x55c4c1479850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c14793d0 0x55c4c1479850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 -- 172.21.15.86:0/3242711787 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4c124b360 con 0x55c4c1478530 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79c3400700 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1478530 0x55c4c1478df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79b00ca880 tx=0x7f79b0096850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcedf54a6150d605 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79b9200700 1 -- 172.21.15.86:0/3242711787 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0099df0 con 0x55c4c1478530 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79b9200700 1 -- 172.21.15.86:0/3242711787 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b009ab90 con 0x55c4c1478530 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.143+0000 7f79b9200700 1 -- 172.21.15.86:0/3242711787 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0099980 con 0x55c4c1478530 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 -- 172.21.15.86:0/3242711787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1478530 msgr2=0x55c4c1478df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1478530 0x55c4c1478df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79b00ca880 tx=0x7f79b0096850 comp rx=0 tx=0).stop 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 -- 172.21.15.86:0/3242711787 shutdown_connections 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c14793d0 0x55c4c1479850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1477bd0 0x55c4c1477ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 --2- 172.21.15.86:0/3242711787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1478530 0x55c4c1478df0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 -- 172.21.15.86:0/3242711787 >> 172.21.15.86:0/3242711787 conn(0x55c4c13ed1d0 msgr2=0x55c4c1472b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 -- 172.21.15.86:0/3242711787 shutdown_connections 2024-03-22T19:10:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.144+0000 7f79de733b80 1 -- 172.21.15.86:0/3242711787 wait complete. 2024-03-22T19:10:15.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.145+0000 7f79de733b80 1 -- start start 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.145+0000 7f79de733b80 1 Processor -- start 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1481710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1481c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1488c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c1365d30 con 0x55c4c1477bd0 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c13659d0 con 0x55c4c1478530 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c124d650 con 0x55c4c14793d0 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79de733b80 10 client.0 _ll_register_callbacks cb 0x55c4c11d1350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1481710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1481710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41846/0 (socket says 172.21.15.86:41846) 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 -- 172.21.15.86:0/3391619013 learned_addr learned my addr 172.21.15.86:0/3391619013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3400700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1481c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c8c00700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1488c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 -- 172.21.15.86:0/3391619013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 msgr2=0x55c4c1488c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1488c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 -- 172.21.15.86:0/3391619013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 msgr2=0x55c4c1481c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1481c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.146+0000 7f79c3e00700 1 -- 172.21.15.86:0/3391619013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c41a13a0 con 0x55c4c1477bd0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79c3e00700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1481710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f79b4011bb0 tx=0x7f79b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7de9d3eb0dd4a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79ad600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79ad600700 1 -- 172.21.15.86:0/3391619013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b401fc50 con 0x55c4c1477bd0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79ad600700 1 -- 172.21.15.86:0/3391619013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f79b401fdb0 con 0x55c4c1477bd0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4c146aa80 con 0x55c4c1477bd0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79ad600700 1 -- 172.21.15.86:0/3391619013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b4028440 con 0x55c4c1477bd0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79de733b80 10 client.14997 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c4c11f7b10 con 0x55c4c1477bd0 2024-03-22T19:10:15.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79ad600700 1 -- 172.21.15.86:0/3391619013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4c146aa80 con 0x55c4c1477bd0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.147+0000 7f79ad600700 1 -- 172.21.15.86:0/3391619013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f79b40325d0 con 0x55c4c1477bd0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79de733b80 10 client.14997 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79de733b80 10 client.14997 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79de733b80 20 client.14997 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79de733b80 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c148bdf0 0x55c4c1491bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79de733b80 4 client.14997 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4c13c26b0 con 0x55c4c148bdf0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79c3400700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c148bdf0 0x55c4c1491bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79ad600700 10 client.14997 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.148+0000 7f79c3400700 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c148bdf0 0x55c4c1491bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=603397aa35878010 server_cookie=472cff3c999f0c98 in_seq=0 out_seq=0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79ad600700 1 -- 172.21.15.86:0/3391619013 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c4c13c26b0 con 0x55c4c148bdf0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79ad600700 10 client.14997 handle_command_reply: tid=0 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 client.14997 shutdown 2024-03-22T19:10:15.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 msgr2=0x55c4c1481710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1481710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f79b4011bb0 tx=0x7f79b40047c0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 shutdown_connections 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c148bdf0 0x55c4c1491bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1488c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1481710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 --2- 172.21.15.86:0/3391619013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1481c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 >> 172.21.15.86:0/3391619013 conn(0x55c4c13ed1d0 msgr2=0x55c4c14721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79ad600700 0 client.14997 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 shutdown_connections 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.149+0000 7f79de733b80 1 -- 172.21.15.86:0/3391619013 wait complete. 2024-03-22T19:10:15.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.150+0000 7f79de733b80 20 client.14997 trim_cache size 0 max 16384 2024-03-22T19:10:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.167+0000 7f79de733b80 1 Processor -- start 2024-03-22T19:10:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.167+0000 7f79de733b80 1 -- start start 2024-03-22T19:10:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79de733b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1477bd0 0x55c4c1486bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79de733b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 0x55c4c14892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79de733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c14793d0 0x55c4c1472ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79de733b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c13c2990 con 0x55c4c1478530 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79de733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c13c6850 con 0x55c4c14793d0 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79de733b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c1368de0 con 0x55c4c1477bd0 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 0x55c4c14892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 0x55c4c14892f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41856/0 (socket says 172.21.15.86:41856) 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 -- 172.21.15.86:0/3225930787 learned_addr learned my addr 172.21.15.86:0/3225930787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3e00700 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1477bd0 0x55c4c1486bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c8c00700 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c14793d0 0x55c4c1472ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 -- 172.21.15.86:0/3225930787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1477bd0 msgr2=0x55c4c1486bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1477bd0 0x55c4c1486bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 -- 172.21.15.86:0/3225930787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c14793d0 msgr2=0x55c4c1472ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c14793d0 0x55c4c1472ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c3400700 1 -- 172.21.15.86:0/3225930787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4c138fe50 con 0x55c4c1478530 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.168+0000 7f79c8c00700 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c14793d0 0x55c4c1472ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.169+0000 7f79c3400700 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 0x55c4c14892f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79b00095c0 tx=0x7f79b000fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1892d3ca3cd232e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.169+0000 7f79b9c00700 1 -- 172.21.15.86:0/3225930787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0098830 con 0x55c4c1478530 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.169+0000 7f79b9c00700 1 -- 172.21.15.86:0/3225930787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b0099df0 con 0x55c4c1478530 2024-03-22T19:10:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.169+0000 7f79b9c00700 1 -- 172.21.15.86:0/3225930787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0099980 con 0x55c4c1478530 2024-03-22T19:10:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.169+0000 7f79de733b80 1 -- 172.21.15.86:0/3225930787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 msgr2=0x55c4c14892f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.169+0000 7f79de733b80 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 0x55c4c14892f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79b00095c0 tx=0x7f79b000fdc0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 -- 172.21.15.86:0/3225930787 shutdown_connections 2024-03-22T19:10:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c14793d0 0x55c4c1472ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c1477bd0 0x55c4c1486bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 --2- 172.21.15.86:0/3225930787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1478530 0x55c4c14892f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 -- 172.21.15.86:0/3225930787 >> 172.21.15.86:0/3225930787 conn(0x55c4c13ed1d0 msgr2=0x55c4c147f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 -- 172.21.15.86:0/3225930787 shutdown_connections 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 -- 172.21.15.86:0/3225930787 wait complete. 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 -- start start 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.170+0000 7f79de733b80 1 Processor -- start 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1482df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1483330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1485c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c1486170 con 0x55c4c1477bd0 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c14862e0 con 0x55c4c1478530 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c14864a0 con 0x55c4c14793d0 2024-03-22T19:10:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79de733b80 10 client.0 _ll_register_callbacks cb 0x7f79b4003710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1483330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1482df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1483330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43210/0 (socket says 172.21.15.86:43210) 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 -- 172.21.15.86:0/764358741 learned_addr learned my addr 172.21.15.86:0/764358741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c8c00700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1485c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 -- 172.21.15.86:0/764358741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 msgr2=0x55c4c1485c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1485c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 -- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 msgr2=0x55c4c1482df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1482df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 -- 172.21.15.86:0/764358741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4c124b360 con 0x55c4c1478530 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3e00700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1482df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79c3400700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1483330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79b0096990 tx=0x7f79b000ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb33c3c77a297190 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.171+0000 7f79ad600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79ad600700 1 -- 172.21.15.86:0/764358741 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0098830 con 0x55c4c1478530 2024-03-22T19:10:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79ad600700 1 -- 172.21.15.86:0/764358741 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f79b0099980 con 0x55c4c1478530 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79b009ab90 con 0x55c4c1478530 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79ad600700 1 -- 172.21.15.86:0/764358741 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b0099b40 con 0x55c4c1478530 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 10 client.25031 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c4c124fc80 con 0x55c4c1478530 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79ad600700 1 -- 172.21.15.86:0/764358741 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b009ab90 con 0x55c4c1478530 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79ad600700 1 -- 172.21.15.86:0/764358741 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f79b0096280 con 0x55c4c1478530 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 10 client.25031 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 10 client.25031 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 20 client.25031 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c14e29e0 0x55c4c14df490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 4 client.25031 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.172+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c4c1218ca0 con 0x55c4c14e29e0 2024-03-22T19:10:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79c3e00700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c14e29e0 0x55c4c14df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79c3e00700 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c14e29e0 0x55c4c14df490 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f28d437666726c0 server_cookie=2c17bbda63fdcd30 in_seq=0 out_seq=0 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79ad600700 10 client.25031 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79ad600700 1 -- 172.21.15.86:0/764358741 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c4c1218ca0 con 0x55c4c14e29e0 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79ad600700 10 client.25031 handle_command_reply: tid=0 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79de733b80 1 client.25031 shutdown 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 msgr2=0x55c4c1483330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79de733b80 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1483330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79b0096990 tx=0x7f79b000ff00 comp rx=0 tx=0).stop 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 shutdown_connections 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.173+0000 7f79de733b80 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55c4c14e29e0 0x55c4c14df490 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4c14793d0 0x55c4c1485c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4c1477bd0 0x55c4c1482df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79ad600700 0 client.25031 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 1 --2- 172.21.15.86:0/764358741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4c1478530 0x55c4c1483330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 >> 172.21.15.86:0/764358741 conn(0x55c4c13ed1d0 msgr2=0x55c4c1472a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 shutdown_connections 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 1 -- 172.21.15.86:0/764358741 wait complete. 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79de733b80 20 client.25031 trim_cache size 0 max 16384 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24557, 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 58.043116845999997, 2024-03-22T19:10:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 69.491421837000004 2024-03-22T19:10:15.181 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79a80a0ba0 msgr2=0x7f79a80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79a80a0ba0 0x7f79a80a3060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79c4101da0 tx=0x7f79b4009260 comp rx=0 tx=0).stop 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4101f40 msgr2=0x7f79c4199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4101f40 0x7f79c4199440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009fd0 tx=0x7f79b000fa00 comp rx=0 tx=0).stop 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 shutdown_connections 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f79a80a0ba0 0x7f79a80a3060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79c41028e0 0x7f79c41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c4100d50 0x7f79c4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 --2- 172.21.15.86:0/1610340635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c4101f40 0x7f79c4199440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.174+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 >> 172.21.15.86:0/1610340635 conn(0x7f79c40fc2c0 msgr2=0x7f79c4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.175+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 shutdown_connections 2024-03-22T19:10:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.175+0000 7f79cac00700 1 -- 172.21.15.86:0/1610340635 wait complete. 2024-03-22T19:10:15.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 -- 172.21.15.86:0/1520490248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 msgr2=0x7fb110108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 --2- 172.21.15.86:0/1520490248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb110108ea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb10800b0a0 tx=0x7fb10800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 -- 172.21.15.86:0/1520490248 shutdown_connections 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 --2- 172.21.15.86:0/1520490248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb110108ea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 --2- 172.21.15.86:0/1520490248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 0x7fb110103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 --2- 172.21.15.86:0/1520490248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb110104100 0x7fb1101064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 -- 172.21.15.86:0/1520490248 >> 172.21.15.86:0/1520490248 conn(0x7fb1100fb1c0 msgr2=0x7fb1100fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 -- 172.21.15.86:0/1520490248 shutdown_connections 2024-03-22T19:10:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.386+0000 7fb117600700 1 -- 172.21.15.86:0/1520490248 wait complete. 2024-03-22T19:10:15.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 Processor -- start 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 -- start start 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 0x7fb110196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb110104100 0x7fb110197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb11019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11019e8f0 con 0x7fb110106a30 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11019ea30 con 0x7fb110104100 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.387+0000 7fb117600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11019ebc0 con 0x7fb1101017d0 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb114c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 0x7fb110196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb11019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb114c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 0x7fb110196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:36476/0 (socket says 172.21.15.86:36476) 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb114c00700 1 -- 172.21.15.86:0/3151711452 learned_addr learned my addr 172.21.15.86:0/3151711452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb10fe00700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb110104100 0x7fb110197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 -- 172.21.15.86:0/3151711452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 msgr2=0x7fb110196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 0x7fb110196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 -- 172.21.15.86:0/3151711452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb110104100 msgr2=0x7fb110197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb110104100 0x7fb110197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 -- 172.21.15.86:0/3151711452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108009d00 con 0x7fb110106a30 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb115600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb11019e320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb108007b90 tx=0x7fb108007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.388+0000 7fb10d600700 1 -- 172.21.15.86:0/3151711452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800e040 con 0x7fb110106a30 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.389+0000 7fb10d600700 1 -- 172.21.15.86:0/3151711452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb10800be40 con 0x7fb110106a30 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.389+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11019ee40 con 0x7fb110106a30 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.389+0000 7fb10d600700 1 -- 172.21.15.86:0/3151711452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10801b460 con 0x7fb110106a30 2024-03-22T19:10:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.389+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11019f200 con 0x7fb110106a30 2024-03-22T19:10:15.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.392+0000 7fb10d600700 1 -- 172.21.15.86:0/3151711452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fb10801d470 con 0x7fb110106a30 2024-03-22T19:10:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.394+0000 7fb10d600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb0f80a0ba0 0x7fb0f80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.394+0000 7fb12b1b7b80 1 Processor -- start 2024-03-22T19:10:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.394+0000 7fb10d600700 1 -- 172.21.15.86:0/3151711452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb1080cd2c0 con 0x7fb110106a30 2024-03-22T19:10:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.394+0000 7fb114c00700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb0f80a0ba0 0x7fb0f80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.394+0000 7fb12b1b7b80 1 -- start start 2024-03-22T19:10:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 0x55894bf75ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf76530 0x55894bf76df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb0f80a0ba0 0x7fb0f80a3060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb104009d20 tx=0x7fb104009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 0x55894bf75ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf77850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bece940 con 0x55894bf773d0 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 0x55894bf75ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:43228/0 (socket says 172.21.15.86:43228) 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 -- 172.21.15.86:0/3731457755 learned_addr learned my addr 172.21.15.86:0/3731457755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894becc300 con 0x55894bf75bd0 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bec7bf0 con 0x55894bf76530 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb10fe00700 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf76530 0x55894bf76df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb115600700 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf77850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 -- 172.21.15.86:0/3731457755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf76530 msgr2=0x55894bf76df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf76530 0x55894bf76df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 -- 172.21.15.86:0/3731457755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 msgr2=0x55894bf77850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf77850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.395+0000 7fb114c00700 1 -- 172.21.15.86:0/3731457755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894bd49360 con 0x55894bf75bd0 2024-03-22T19:10:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.396+0000 7fb114c00700 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 0x55894bf75ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e570 tx=0x7fb104014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9525caaaf3a47af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.396+0000 7fb101c00700 1 -- 172.21.15.86:0/3731457755 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004830 con 0x55894bf75bd0 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.396+0000 7fb101c00700 1 -- 172.21.15.86:0/3731457755 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb104021070 con 0x55894bf75bd0 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.396+0000 7fb101c00700 1 -- 172.21.15.86:0/3731457755 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10401d410 con 0x55894bf75bd0 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 msgr2=0x55894bf75ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 0x55894bf75ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e570 tx=0x7fb104014000 comp rx=0 tx=0).stop 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 shutdown_connections 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf77850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf75bd0 0x55894bf75ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/3731457755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf76530 0x55894bf76df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 >> 172.21.15.86:0/3731457755 conn(0x55894beeb1d0 msgr2=0x55894bf70b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 shutdown_connections 2024-03-22T19:10:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/3731457755 wait complete. 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 -- start start 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.397+0000 7fb12b1b7b80 1 Processor -- start 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf7fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf7ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf86fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894be63d30 con 0x55894bf773d0 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894be639d0 con 0x55894bf76530 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bd4b650 con 0x55894bf75bd0 2024-03-22T19:10:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb12b1b7b80 10 client.0 _ll_register_callbacks cb 0x55894bccf350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb115600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf86fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb114c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf7fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb115600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf86fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41902/0 (socket says 172.21.15.86:41902) 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb115600700 1 -- 172.21.15.86:0/1910939454 learned_addr learned my addr 172.21.15.86:0/1910939454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb10fe00700 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf7ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb114c00700 1 -- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 msgr2=0x55894bf7ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.398+0000 7fb114c00700 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf7ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb114c00700 1 -- 172.21.15.86:0/1910939454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 msgr2=0x55894bf86fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb114c00700 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf86fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb114c00700 1 -- 172.21.15.86:0/1910939454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb104017070 con 0x55894bf75bd0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb114c00700 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf7fa50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb10400eb60 tx=0x7fb104005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f8010d3a3ca816 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb0f6000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb0f6000700 1 -- 172.21.15.86:0/1910939454 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004b60 con 0x55894bf75bd0 2024-03-22T19:10:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb0f6000700 1 -- 172.21.15.86:0/1910939454 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb104004cc0 con 0x55894bf75bd0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55894bf68a80 con 0x55894bf75bd0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb0f6000700 1 -- 172.21.15.86:0/1910939454 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb10400ee10 con 0x55894bf75bd0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb12b1b7b80 10 client.25048 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.399+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55894bcf5b10 con 0x55894bf75bd0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb0f6000700 1 -- 172.21.15.86:0/1910939454 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55894bf68a80 con 0x55894bf75bd0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb0f6000700 1 -- 172.21.15.86:0/1910939454 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb104030440 con 0x55894bf75bd0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb12b1b7b80 10 client.25048 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb12b1b7b80 10 client.25048 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb12b1b7b80 20 client.25048 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bf8aad0 0x55894bf8fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb12b1b7b80 4 client.25048 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55894bec06b0 con 0x55894bf8aad0 2024-03-22T19:10:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.400+0000 7fb10fe00700 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bf8aad0 0x55894bf8fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.401+0000 7fb10fe00700 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bf8aad0 0x55894bf8fdb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd90b52e588dfa94 server_cookie=e031dd36d2a08d5b in_seq=0 out_seq=0 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.401+0000 7fb0f6000700 10 client.25048 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.401+0000 7fb0f6000700 1 -- 172.21.15.86:0/1910939454 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55894bec06b0 con 0x55894bf8aad0 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.401+0000 7fb0f6000700 10 client.25048 handle_command_reply: tid=0 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 client.25048 shutdown 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 msgr2=0x55894bf7fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf7fa50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb10400eb60 tx=0x7fb104005730 comp rx=0 tx=0).stop 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 shutdown_connections 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bf8aad0 0x55894bf8fdb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf773d0 0x55894bf86fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf7fa50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1910939454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf7ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb0f6000700 0 client.25048 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 >> 172.21.15.86:0/1910939454 conn(0x55894beeb1d0 msgr2=0x55894bf701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 shutdown_connections 2024-03-22T19:10:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1910939454 wait complete. 2024-03-22T19:10:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.402+0000 7fb12b1b7b80 20 client.25048 trim_cache size 0 max 16384 2024-03-22T19:10:15.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 Processor -- start 2024-03-22T19:10:15.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 -- start start 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf84f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 0x55894bf87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf773d0 0x55894bf777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bec0990 con 0x55894bf76530 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bec4850 con 0x55894bf773d0 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894be66de0 con 0x55894bf75bd0 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb10fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 0x55894bf87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb10fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 0x55894bf87680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41912/0 (socket says 172.21.15.86:41912) 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.420+0000 7fb10fe00700 1 -- 172.21.15.86:0/1219888633 learned_addr learned my addr 172.21.15.86:0/1219888633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb114c00700 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf84f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb115600700 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf773d0 0x55894bf777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb10fe00700 1 -- 172.21.15.86:0/1219888633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 msgr2=0x55894bf84f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb10fe00700 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf84f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb10fe00700 1 -- 172.21.15.86:0/1219888633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf773d0 msgr2=0x55894bf777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb10fe00700 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf773d0 0x55894bf777b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb10fe00700 1 -- 172.21.15.86:0/1219888633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894be8de50 con 0x55894bf76530 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb10fe00700 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 0x55894bf87680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8664ba696483f2bf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb102600700 1 -- 172.21.15.86:0/1219888633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc017070 con 0x55894bf76530 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb102600700 1 -- 172.21.15.86:0/1219888633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0fc018040 con 0x55894bf76530 2024-03-22T19:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.421+0000 7fb102600700 1 -- 172.21.15.86:0/1219888633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0107b0 con 0x55894bf76530 2024-03-22T19:10:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1219888633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 msgr2=0x55894bf87680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 0x55894bf87680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc009260 comp rx=0 tx=0).stop 2024-03-22T19:10:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1219888633 shutdown_connections 2024-03-22T19:10:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf773d0 0x55894bf777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf75bd0 0x55894bf84f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/1219888633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf76530 0x55894bf87680 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1219888633 >> 172.21.15.86:0/1219888633 conn(0x55894beeb1d0 msgr2=0x55894bf7ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1219888633 shutdown_connections 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/1219888633 wait complete. 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.422+0000 7fb12b1b7b80 1 -- start start 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 Processor -- start 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 0x55894bf83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf773d0 0x55894bf84a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bf85140 con 0x55894bf75bd0 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bf852b0 con 0x55894bf76530 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894bfdd090 con 0x55894bf773d0 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb12b1b7b80 10 client.0 _ll_register_callbacks cb 0x7fb0fc009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb114c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 0x55894bf83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb114c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 0x55894bf83f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41914/0 (socket says 172.21.15.86:41914) 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb114c00700 1 -- 172.21.15.86:0/2820092018 learned_addr learned my addr 172.21.15.86:0/2820092018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb10fe00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.423+0000 7fb115600700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf773d0 0x55894bf84a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb114c00700 1 -- 172.21.15.86:0/2820092018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf773d0 msgr2=0x55894bf84a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb114c00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf773d0 0x55894bf84a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb114c00700 1 -- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 msgr2=0x55894bf844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb114c00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf844d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb114c00700 1 -- 172.21.15.86:0/2820092018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc004040 con 0x55894bf75bd0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb10fe00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb114c00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 0x55894bf83f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb104016040 tx=0x7fb104009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2764237ba11ceb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb0f6000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb0f6000700 1 -- 172.21.15.86:0/2820092018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004b70 con 0x55894bf75bd0 2024-03-22T19:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb104021070 con 0x55894bf75bd0 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb0f6000700 1 -- 172.21.15.86:0/2820092018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb104004cd0 con 0x55894bf75bd0 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb0f6000700 1 -- 172.21.15.86:0/2820092018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb104004e90 con 0x55894bf75bd0 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb12b1b7b80 10 client.15021 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55894bd4dc80 con 0x55894bf75bd0 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.424+0000 7fb0f6000700 1 -- 172.21.15.86:0/2820092018 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb104021070 con 0x55894bf75bd0 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb0f6000700 1 -- 172.21.15.86:0/2820092018 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb10400e880 con 0x55894bf75bd0 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb12b1b7b80 10 client.15021 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb12b1b7b80 10 client.15021 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb12b1b7b80 20 client.15021 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bfe1b60 0x55894bfde480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb12b1b7b80 4 client.15021 mds_command: new command op to 24569 tid=0[{"prefix": "status"}] 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55894bd16ca0 con 0x55894bfe1b60 2024-03-22T19:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.425+0000 7fb10fe00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bfe1b60 0x55894bfde480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb10fe00700 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bfe1b60 0x55894bfde480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bed147064089d9 server_cookie=da33f3cca8c9d092 in_seq=0 out_seq=0 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb0f6000700 10 client.15021 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb0f6000700 1 -- 172.21.15.86:0/2820092018 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55894bd16ca0 con 0x55894bfe1b60 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb0f6000700 10 client.15021 handle_command_reply: tid=0 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 client.15021 shutdown 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 msgr2=0x55894bf83f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 0x55894bf83f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb104016040 tx=0x7fb104009340 comp rx=0 tx=0).stop 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 shutdown_connections 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55894bfe1b60 0x55894bfde480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55894bf773d0 0x55894bf84a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55894bf75bd0 0x55894bf83f90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 --2- 172.21.15.86:0/2820092018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55894bf76530 0x55894bf844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb0f6000700 0 client.15021 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 >> 172.21.15.86:0/2820092018 conn(0x55894beeb1d0 msgr2=0x55894bf7ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 shutdown_connections 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.426+0000 7fb12b1b7b80 1 -- 172.21.15.86:0/2820092018 wait complete. 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb12b1b7b80 20 client.15021 trim_cache size 0 max 16384 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24569, 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 61.804205295999999, 2024-03-22T19:10:15.432 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:15.433 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:15.433 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:15.433 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 68.020655089000002 2024-03-22T19:10:15.433 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb0f80a0ba0 msgr2=0x7fb0f80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb0f80a0ba0 0x7fb0f80a3060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb104009d20 tx=0x7fb104009260 comp rx=0 tx=0).stop 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 msgr2=0x7fb11019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb11019e320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb108007b90 tx=0x7fb108007930 comp rx=0 tx=0).stop 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 shutdown_connections 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb0f80a0ba0 0x7fb0f80a3060 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb110106a30 0x7fb11019e320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1101017d0 0x7fb110196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 --2- 172.21.15.86:0/3151711452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb110104100 0x7fb110197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 >> 172.21.15.86:0/3151711452 conn(0x7fb1100fb1c0 msgr2=0x7fb110102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 shutdown_connections 2024-03-22T19:10:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:15.427+0000 7fb117600700 1 -- 172.21.15.86:0/3151711452 wait complete. 2024-03-22T19:10:15.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:15 smithi086 ceph-mon[38032]: pgmap v264: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:15.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:10:15.444 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:10:15.444 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:15.446 INFO:tasks.cephadm:Setup complete, yielding 2024-03-22T19:10:15.446 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T19:10:15.456 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T19:10:15.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-22T19:10:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:15 smithi131 ceph-mon[78559]: pgmap v264: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:10:15.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:10:15.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:15 smithi167 ceph-mon[47728]: pgmap v264: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:10:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:10:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:10:17.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 -- 172.21.15.86:0/4073512693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68102980 msgr2=0x7fce68109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 --2- 172.21.15.86:0/4073512693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68102980 0x7fce68109230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce6000b0d0 tx=0x7fce6000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 -- 172.21.15.86:0/4073512693 shutdown_connections 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 --2- 172.21.15.86:0/4073512693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68102980 0x7fce68109230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 --2- 172.21.15.86:0/4073512693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce68100de0 0x7fce68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 --2- 172.21.15.86:0/4073512693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68101fe0 0x7fce68102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.061+0000 7fce70000700 1 -- 172.21.15.86:0/4073512693 >> 172.21.15.86:0/4073512693 conn(0x7fce680fc380 msgr2=0x7fce680fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.062+0000 7fce70000700 1 -- 172.21.15.86:0/4073512693 shutdown_connections 2024-03-22T19:10:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.062+0000 7fce70000700 1 -- 172.21.15.86:0/4073512693 wait complete. 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.062+0000 7fce70000700 1 Processor -- start 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 -- start start 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce68100de0 0x7fce68199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 0x7fce68199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68102980 0x7fce681a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce681a0b50 con 0x7fce68101fe0 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce681a0c90 con 0x7fce68100de0 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce70000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce681a0e20 con 0x7fce68102980 2024-03-22T19:10:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce6cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 0x7fce68199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce6e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68102980 0x7fce681a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce6cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 0x7fce68199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41930/0 (socket says 172.21.15.86:41930) 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce6cc00700 1 -- 172.21.15.86:0/1225322437 learned_addr learned my addr 172.21.15.86:0/1225322437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.063+0000 7fce6d600700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce68100de0 0x7fce68199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6e000700 1 -- 172.21.15.86:0/1225322437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce68100de0 msgr2=0x7fce68199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6e000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce68100de0 0x7fce68199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6e000700 1 -- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 msgr2=0x7fce68199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6e000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 0x7fce68199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6e000700 1 -- 172.21.15.86:0/1225322437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60009d00 con 0x7fce68102980 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6cc00700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 0x7fce68199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce6e000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68102980 0x7fce681a0580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce6000b770 tx=0x7fce60008e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000e040 con 0x7fce68102980 2024-03-22T19:10:17.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce681a10a0 con 0x7fce68102980 2024-03-22T19:10:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce60005070 con 0x7fce68102980 2024-03-22T19:10:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6001d6d0 con 0x7fce68102980 2024-03-22T19:10:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.064+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce681a14c0 con 0x7fce68102980 2024-03-22T19:10:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.065+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6804ea80 con 0x7fce68102980 2024-03-22T19:10:17.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.067+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fce60005450 con 0x7fce68102980 2024-03-22T19:10:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.068+0000 7fce66000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fce500a0b20 0x7fce500a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.069+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fce600cd630 con 0x7fce68102980 2024-03-22T19:10:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.069+0000 7fce6d600700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fce500a0b20 0x7fce500a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.069+0000 7fce6d600700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fce500a0b20 0x7fce500a2fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce68101e40 tx=0x7fce5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:17.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.071+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fce60096fb0 con 0x7fce68102980 2024-03-22T19:10:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.293+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce681a16e0 con 0x7fce500a0b20 2024-03-22T19:10:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.294+0000 7fce66000700 1 -- 172.21.15.86:0/1225322437 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fce681a16e0 con 0x7fce500a0b20 2024-03-22T19:10:17.295 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-03-22T19:10:17.295 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-03-22T19:10:17.295 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-03-22T19:10:17.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fce500a0b20 msgr2=0x7fce500a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:17.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fce500a0b20 0x7fce500a2fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce68101e40 tx=0x7fce5c009260 comp rx=0 tx=0).stop 2024-03-22T19:10:17.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68102980 msgr2=0x7fce681a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68102980 0x7fce681a0580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce6000b770 tx=0x7fce60008e90 comp rx=0 tx=0).stop 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 shutdown_connections 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fce500a0b20 0x7fce500a2fe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce68102980 0x7fce681a0580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce68100de0 0x7fce68199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 --2- 172.21.15.86:0/1225322437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce68101fe0 0x7fce68199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 >> 172.21.15.86:0/1225322437 conn(0x7fce680fc380 msgr2=0x7fce68072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 shutdown_connections 2024-03-22T19:10:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:17.297+0000 7fce70000700 1 -- 172.21.15.86:0/1225322437 wait complete. 2024-03-22T19:10:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:17 smithi131 ceph-mon[78559]: pgmap v265: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:17.478 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:17 smithi086 ceph-mon[38032]: pgmap v265: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:17 smithi167 ceph-mon[47728]: pgmap v265: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:17.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-22T19:10:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:18 smithi131 ceph-mon[78559]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:18 smithi086 ceph-mon[38032]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:18 smithi167 ceph-mon[47728]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:19.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 -- 172.21.15.86:0/1825658295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed0074d80 msgr2=0x7f4ed00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1825658295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed0074d80 0x7f4ed00731e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009990 tx=0x7f4ec0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 -- 172.21.15.86:0/1825658295 shutdown_connections 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1825658295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed0108cc0 0x7f4ed010b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1825658295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed0074d80 0x7f4ed00731e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1825658295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ed00737b0 0x7f4ed0073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 -- 172.21.15.86:0/1825658295 >> 172.21.15.86:0/1825658295 conn(0x7f4ed00fc380 msgr2=0x7f4ed00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 -- 172.21.15.86:0/1825658295 shutdown_connections 2024-03-22T19:10:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.214+0000 7f4ed6200700 1 -- 172.21.15.86:0/1825658295 wait complete. 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 Processor -- start 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 -- start start 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 0x7f4ed0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed0074d80 0x7f4ed0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ed0108cc0 0x7f4ed01a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01a0a70 con 0x7f4ed00737b0 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01a0bb0 con 0x7f4ed0108cc0 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.216+0000 7f4ed6200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01a0d40 con 0x7f4ed0074d80 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 0x7f4ed0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 0x7f4ed0198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34096/0 (socket says 172.21.15.86:34096) 2024-03-22T19:10:19.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 -- 172.21.15.86:0/1455732106 learned_addr learned my addr 172.21.15.86:0/1455732106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecea00700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed0074d80 0x7f4ed0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecfe00700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ed0108cc0 0x7f4ed01a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 -- 172.21.15.86:0/1455732106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed0074d80 msgr2=0x7f4ed0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed0074d80 0x7f4ed0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 -- 172.21.15.86:0/1455732106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ed0108cc0 msgr2=0x7f4ed01a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ed0108cc0 0x7f4ed01a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 -- 172.21.15.86:0/1455732106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00095f0 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.217+0000 7f4ecf400700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 0x7f4ed0198f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0005910 tx=0x7f4ec0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.218+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec001d070 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.218+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec0022470 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.218+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed01a0fc0 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.218+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0017650 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.218+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed01a1320 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.219+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed0107ad0 con 0x7f4ed00737b0 2024-03-22T19:10:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.220+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f4ec000fad0 con 0x7f4ed00737b0 2024-03-22T19:10:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.222+0000 7f4ec7e00700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4eb40a0b20 0x7f4eb40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.222+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f4ec00cd490 con 0x7f4ed00737b0 2024-03-22T19:10:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.222+0000 7f4ecea00700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4eb40a0b20 0x7f4eb40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.222+0000 7f4ecea00700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4eb40a0b20 0x7f4eb40a2fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009c60 tx=0x7f4ebc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.226+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ec0096d80 con 0x7f4ed00737b0 2024-03-22T19:10:19.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:19 smithi086 ceph-mon[38032]: pgmap v266: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:19.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.438+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed00611f0 con 0x7f4eb40a0b20 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.451+0000 7f4ec7e00700 1 -- 172.21.15.86:0/1455732106 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f4ed00611f0 con 0x7f4eb40a0b20 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (74s) 5s ago 73s 18.1M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 7b0ff2dfbc7f 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi131 running (72s) 6s ago 71s 30.1M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb e0e448232279 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi167 running (82s) 6s ago 82s 22.0M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb e0a637683c29 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (88s) 5s ago 87s 13.8M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 1cc41c54c2fd 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi131 running (86s) 6s ago 85s 18.0M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 1877d9779774 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi167 running (84s) 6s ago 84s 18.4M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 8d33218c115e 2024-03-22T19:10:19.452 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (76s) 5s ago 76s 17.2M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb d2801444548f 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi131 running (81s) 6s ago 80s 20.0M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 4ab593a95dda 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi167 running (79s) 6s ago 78s 18.8M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 8661c4f0c14c 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (90s) 5s ago 90s 13.8M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb cb2c14ab8a5a 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi131 running (92s) 6s ago 92s 19.8M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 7637362f1446 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi167 running (94s) 6s ago 94s 17.3M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 67ecf6a5a307 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (7m) 5s ago 7m 533M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 104e8310a3b2 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi131 *:8443,8765 running (5m) 6s ago 5m 459M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb 33c0402d0c6c 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi167 *:8443,8765 running (5m) 6s ago 5m 458M - 18.2.2-819-g899aeb57 b3d9fcfc8fbb a4e12fb0f443 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (7m) 5s ago 7m 93.1M 2048M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 0e4d0f4d2b91 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi131 running (5m) 6s ago 5m 66.0M 2048M 18.2.2-819-g899aeb57 b3d9fcfc8fbb d1576d41fad9 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi167 running (5m) 6s ago 5m 69.2M 2048M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 842768e7fcbb 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (5m) 5s ago 5m 66.0M 4317M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 5abf4e4ca838 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi131 running (4m) 6s ago 4m 72.6M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb b3b28ab73e26 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi167 running (4m) 6s ago 4m 70.6M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 1488ed471f3e 2024-03-22T19:10:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (4m) 5s ago 4m 60.8M 4317M 18.2.2-819-g899aeb57 b3d9fcfc8fbb ecb52ae5cc6c 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi131 running (4m) 6s ago 4m 61.7M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 66a6567fdb5a 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi167 running (3m) 6s ago 3m 62.0M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 2f2037a1db68 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (3m) 5s ago 3m 56.7M 4317M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 75e54cc65ac3 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi131 running (3m) 6s ago 3m 62.8M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 671a045238ab 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi167 running (2m) 6s ago 2m 62.7M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 7d113e082d42 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (2m) 5s ago 2m 54.7M 4317M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 81211fbdf23f 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi131 running (2m) 6s ago 2m 61.3M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 3f3fa526890f 2024-03-22T19:10:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi167 running (111s) 6s ago 111s 50.8M 4278M 18.2.2-819-g899aeb57 b3d9fcfc8fbb 68a9b8a14525 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4eb40a0b20 msgr2=0x7f4eb40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4eb40a0b20 0x7f4eb40a2fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009c60 tx=0x7f4ebc009260 comp rx=0 tx=0).stop 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 msgr2=0x7f4ed0198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 0x7f4ed0198f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0005910 tx=0x7f4ec0009580 comp rx=0 tx=0).stop 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 shutdown_connections 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4eb40a0b20 0x7f4eb40a2fe0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ed0108cc0 0x7f4ed01a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed00737b0 0x7f4ed0198f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 --2- 172.21.15.86:0/1455732106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed0074d80 0x7f4ed0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 >> 172.21.15.86:0/1455732106 conn(0x7f4ed00fc380 msgr2=0x7f4ed0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 shutdown_connections 2024-03-22T19:10:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:19.454+0000 7f4ed6200700 1 -- 172.21.15.86:0/1455732106 wait complete. 2024-03-22T19:10:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:19 smithi131 ceph-mon[78559]: pgmap v266: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:19 smithi167 ceph-mon[47728]: pgmap v266: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:19.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-22T19:10:20.219 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:20.418 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:20 smithi086 ceph-mon[38032]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:20.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:20 smithi131 ceph-mon[78559]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:20 smithi167 ceph-mon[47728]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.380+0000 7f899fa00700 1 -- 172.21.15.86:0/3506745267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998100de0 msgr2=0x7f8998101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.380+0000 7f899fa00700 1 --2- 172.21.15.86:0/3506745267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998100de0 0x7f8998101200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f898c0099c0 tx=0x7f898c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 -- 172.21.15.86:0/3506745267 shutdown_connections 2024-03-22T19:10:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 --2- 172.21.15.86:0/3506745267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998102980 0x7f8998109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 --2- 172.21.15.86:0/3506745267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998100de0 0x7f8998101200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 --2- 172.21.15.86:0/3506745267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 0x7f8998102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 -- 172.21.15.86:0/3506745267 >> 172.21.15.86:0/3506745267 conn(0x7f89980fc380 msgr2=0x7f89980fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 -- 172.21.15.86:0/3506745267 shutdown_connections 2024-03-22T19:10:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.381+0000 7f899fa00700 1 -- 172.21.15.86:0/3506745267 wait complete. 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 Processor -- start 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 -- start start 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998100de0 0x7f8998074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 0x7f89980731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 0x7f89980736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998073dc0 con 0x7f8998102980 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998073f30 con 0x7f8998101fe0 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.382+0000 7f899fa00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89981a0df0 con 0x7f8998100de0 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 0x7f89980736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 0x7f89980736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34116/0 (socket says 172.21.15.86:34116) 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 -- 172.21.15.86:0/2658064355 learned_addr learned my addr 172.21.15.86:0/2658064355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f8997e00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 0x7f89980731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899d000700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998100de0 0x7f8998074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 -- 172.21.15.86:0/2658064355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998100de0 msgr2=0x7f8998074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998100de0 0x7f8998074ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 -- 172.21.15.86:0/2658064355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 msgr2=0x7f89980731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 0x7f89980731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 -- 172.21.15.86:0/2658064355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0095f0 con 0x7f8998102980 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f8997e00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 0x7f89980731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.383+0000 7f899da00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 0x7f89980736e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8990007ee0 tx=0x7f899000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.384+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990017070 con 0x7f8998102980 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.384+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899000f040 con 0x7f8998102980 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.384+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89981a0ff0 con 0x7f8998102980 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.384+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990005580 con 0x7f8998102980 2024-03-22T19:10:21.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.384+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89981a1510 con 0x7f8998102980 2024-03-22T19:10:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.385+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8998108260 con 0x7f8998102980 2024-03-22T19:10:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.386+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f899000db20 con 0x7f8998102980 2024-03-22T19:10:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.388+0000 7f8995600700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f89800a0ba0 0x7f89800a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.388+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f89900cb860 con 0x7f8998102980 2024-03-22T19:10:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.388+0000 7f899d000700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f89800a0ba0 0x7f89800a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:21.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.388+0000 7f899d000700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f89800a0ba0 0x7f89800a3060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f898c009990 tx=0x7f898c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:21.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.392+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f89900950c0 con 0x7f8998102980 2024-03-22T19:10:21.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:21 smithi131 ceph-mon[78559]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:21.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:21 smithi086 ceph-mon[38032]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:21 smithi167 ceph-mon[47728]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:21.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.606+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f89980611f0 con 0x7f89800a0ba0 2024-03-22T19:10:21.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.612+0000 7f8995600700 1 -- 172.21.15.86:0/2658064355 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f89980611f0 con 0x7f89800a0ba0 2024-03-22T19:10:21.612 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-22T19:10:21.612 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 8s ago 99s smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-03-22T19:10:21.612 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 8s ago 5m smithi086=x;smithi131=y;smithi167=z;count:3 2024-03-22T19:10:21.612 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 8s ago 6m smithi086:172.21.15.86=a;smithi131:172.21.15.131=b;smithi167:172.21.15.167=c;count:3 2024-03-22T19:10:21.612 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 8s ago - 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f89800a0ba0 msgr2=0x7f89800a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f89800a0ba0 0x7f89800a3060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f898c009990 tx=0x7f898c000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 msgr2=0x7f89980736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 0x7f89980736e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8990007ee0 tx=0x7f899000d3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 shutdown_connections 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f89800a0ba0 0x7f89800a3060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8998102980 0x7f89980736e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998100de0 0x7f8998074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 --2- 172.21.15.86:0/2658064355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8998101fe0 0x7f89980731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.614+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 >> 172.21.15.86:0/2658064355 conn(0x7f89980fc380 msgr2=0x7f89981053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.615+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 shutdown_connections 2024-03-22T19:10:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:21.615+0000 7f899fa00700 1 -- 172.21.15.86:0/2658064355 wait complete. 2024-03-22T19:10:22.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-22T19:10:22.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:22 smithi086 ceph-mon[38032]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:22.386 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:22.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:22 smithi131 ceph-mon[78559]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:22 smithi167 ceph-mon[47728]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:23.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:23 smithi086 ceph-mon[38032]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:23.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:23 smithi131 ceph-mon[78559]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:23 smithi167 ceph-mon[47728]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 -- 172.21.15.86:0/3314423271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102980 msgr2=0x7fb558109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 --2- 172.21.15.86:0/3314423271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102980 0x7fb558109230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb55000b0d0 tx=0x7fb55000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 -- 172.21.15.86:0/3314423271 shutdown_connections 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 --2- 172.21.15.86:0/3314423271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102980 0x7fb558109230 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 --2- 172.21.15.86:0/3314423271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 --2- 172.21.15.86:0/3314423271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558101fe0 0x7fb558102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 -- 172.21.15.86:0/3314423271 >> 172.21.15.86:0/3314423271 conn(0x7fb5580fc380 msgr2=0x7fb5580fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:23.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.627+0000 7fb55e600700 1 -- 172.21.15.86:0/3314423271 shutdown_connections 2024-03-22T19:10:23.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.628+0000 7fb55e600700 1 -- 172.21.15.86:0/3314423271 wait complete. 2024-03-22T19:10:23.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.629+0000 7fb55e600700 1 Processor -- start 2024-03-22T19:10:23.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.629+0000 7fb55e600700 1 -- start start 2024-03-22T19:10:23.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb55e600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:23.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb55e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558101fe0 0x7fb558199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:23.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb55e600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558102980 0x7fb5581a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:23.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb55e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581a0b50 con 0x7fb558101fe0 2024-03-22T19:10:23.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb55e600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581a0c90 con 0x7fb558100de0 2024-03-22T19:10:23.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb55e600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581a0e20 con 0x7fb558102980 2024-03-22T19:10:23.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:23.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb556a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558101fe0 0x7fb558199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34996/0 (socket says 172.21.15.86:34996) 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 -- 172.21.15.86:0/564814810 learned_addr learned my addr 172.21.15.86:0/564814810 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb556a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558101fe0 0x7fb558199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34138/0 (socket says 172.21.15.86:34138) 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557e00700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558102980 0x7fb5581a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 -- 172.21.15.86:0/564814810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558102980 msgr2=0x7fb5581a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558102980 0x7fb5581a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 -- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558101fe0 msgr2=0x7fb558199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558101fe0 0x7fb558199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557400700 1 -- 172.21.15.86:0/564814810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550009d00 con 0x7fb558100de0 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.630+0000 7fb557e00700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558102980 0x7fb5581a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.631+0000 7fb557400700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558199010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb54800e970 tx=0x7fb54800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.631+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548009790 con 0x7fb558100de0 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.631+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54800eec0 con 0x7fb558100de0 2024-03-22T19:10:23.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.631+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548003ce0 con 0x7fb558100de0 2024-03-22T19:10:23.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.631+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5581a1100 con 0x7fb558100de0 2024-03-22T19:10:23.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.631+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5581a15b0 con 0x7fb558100de0 2024-03-22T19:10:23.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.632+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55804ea80 con 0x7fb558100de0 2024-03-22T19:10:23.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.634+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fb54800ccd0 con 0x7fb558100de0 2024-03-22T19:10:23.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.635+0000 7fb54fe00700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb53c0a0ba0 0x7fb53c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.635+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb548014070 con 0x7fb558100de0 2024-03-22T19:10:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.635+0000 7fb556a00700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb53c0a0ba0 0x7fb53c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.636+0000 7fb556a00700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb53c0a0ba0 0x7fb53c0a3060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb544009c60 tx=0x7fb544009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.638+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb548095d10 con 0x7fb558100de0 2024-03-22T19:10:23.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.845+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5581a1830 con 0x7fb53c0a0ba0 2024-03-22T19:10:23.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.847+0000 7fb54fe00700 1 -- 172.21.15.86:0/564814810 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fb5581a1830 con 0x7fb53c0a0ba0 2024-03-22T19:10:23.848 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-03-22T19:10:23.848 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-03-22T19:10:23.848 INFO:teuthology.orchestra.run.smithi086.stdout:smithi131 172.21.15.131 2024-03-22T19:10:23.849 INFO:teuthology.orchestra.run.smithi086.stdout:smithi167 172.21.15.167 2024-03-22T19:10:23.849 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb53c0a0ba0 msgr2=0x7fb53c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb53c0a0ba0 0x7fb53c0a3060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb544009c60 tx=0x7fb544009260 comp rx=0 tx=0).stop 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 msgr2=0x7fb558199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558199010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb54800e970 tx=0x7fb54800ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 shutdown_connections 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb53c0a0ba0 0x7fb53c0a3060 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb558102980 0x7fb5581a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb558100de0 0x7fb558199010 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 --2- 172.21.15.86:0/564814810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558101fe0 0x7fb558199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.849+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 >> 172.21.15.86:0/564814810 conn(0x7fb5580fc380 msgr2=0x7fb558072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:23.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.850+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 shutdown_connections 2024-03-22T19:10:23.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:23.850+0000 7fb55e600700 1 -- 172.21.15.86:0/564814810 wait complete. 2024-03-22T19:10:24.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-22T19:10:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:25 smithi131 ceph-mon[78559]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:25.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:25 smithi131 ceph-mon[78559]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:25 smithi086 ceph-mon[38032]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:25.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:25 smithi086 ceph-mon[38032]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:25 smithi167 ceph-mon[47728]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:25 smithi167 ceph-mon[47728]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 -- 172.21.15.86:0/1894380674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 msgr2=0x7f3198073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 --2- 172.21.15.86:0/1894380674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f3198073240 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3188009990 tx=0x7f3188009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 -- 172.21.15.86:0/1894380674 shutdown_connections 2024-03-22T19:10:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 --2- 172.21.15.86:0/1894380674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3198108cb0 0x7f319810b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 --2- 172.21.15.86:0/1894380674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f3198073240 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 --2- 172.21.15.86:0/1894380674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198073810 0x7f3198073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 -- 172.21.15.86:0/1894380674 >> 172.21.15.86:0/1894380674 conn(0x7f31980fc380 msgr2=0x7f31980fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 -- 172.21.15.86:0/1894380674 shutdown_connections 2024-03-22T19:10:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.890+0000 7f319de00700 1 -- 172.21.15.86:0/1894380674 wait complete. 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.891+0000 7f319de00700 1 Processor -- start 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 -- start start 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3198073810 0x7f3198198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f31981994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198108cb0 0x7f31981a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31981a0a60 con 0x7f3198074de0 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31981a0ba0 con 0x7f3198073810 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f319de00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31981a0d30 con 0x7f3198108cb0 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f3196000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f31981994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f3196000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f31981994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34162/0 (socket says 172.21.15.86:34162) 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f3196000700 1 -- 172.21.15.86:0/3078511296 learned_addr learned my addr 172.21.15.86:0/3078511296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f3196a00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3198073810 0x7f3198198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.892+0000 7f3197400700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198108cb0 0x7f31981a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3196000700 1 -- 172.21.15.86:0/3078511296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198108cb0 msgr2=0x7f31981a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3196000700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198108cb0 0x7f31981a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3196000700 1 -- 172.21.15.86:0/3078511296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3198073810 msgr2=0x7f3198198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3196000700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3198073810 0x7f3198198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3196000700 1 -- 172.21.15.86:0/3078511296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880095f0 con 0x7f3198074de0 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3197400700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198108cb0 0x7f31981a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f3196000700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f31981994b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9a0 tx=0x7f318400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184009790 con 0x7f3198074de0 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31840044e0 con 0x7f3198074de0 2024-03-22T19:10:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31981a0fb0 con 0x7f3198074de0 2024-03-22T19:10:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184003d20 con 0x7f3198074de0 2024-03-22T19:10:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.893+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31981a1480 con 0x7f3198074de0 2024-03-22T19:10:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.894+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3198107a70 con 0x7f3198074de0 2024-03-22T19:10:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.898+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f3184003e80 con 0x7f3198074de0 2024-03-22T19:10:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.899+0000 7f319ca00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f317c0a0b20 0x7f317c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:25.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.899+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3184014070 con 0x7f3198074de0 2024-03-22T19:10:25.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.899+0000 7f3196a00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f317c0a0b20 0x7f317c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:25.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.900+0000 7f3196a00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f317c0a0b20 0x7f317c0a2fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f318800b680 tx=0x7f3188005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:25.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:25.901+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3184095d50 con 0x7f3198074de0 2024-03-22T19:10:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.112+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f31980611f0 con 0x7f317c0a0b20 2024-03-22T19:10:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.114+0000 7f319ca00700 1 -- 172.21.15.86:0/3078511296 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f31980611f0 con 0x7f317c0a0b20 2024-03-22T19:10:26.116 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-22T19:10:26.116 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-22T19:10:26.116 INFO:teuthology.orchestra.run.smithi086.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-22T19:10:26.116 INFO:teuthology.orchestra.run.smithi086.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 112s ago Has a FileSystem, LVM detected 2024-03-22T19:10:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f317c0a0b20 msgr2=0x7f317c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f317c0a0b20 0x7f317c0a2fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f318800b680 tx=0x7f3188005fd0 comp rx=0 tx=0).stop 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 msgr2=0x7f31981994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f31981994b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9a0 tx=0x7f318400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 shutdown_connections 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f317c0a0b20 0x7f317c0a2fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3198108cb0 0x7f31981a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3198073810 0x7f3198198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 --2- 172.21.15.86:0/3078511296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198074de0 0x7f31981994b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 >> 172.21.15.86:0/3078511296 conn(0x7f31980fc380 msgr2=0x7f3198104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 shutdown_connections 2024-03-22T19:10:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:26.117+0000 7f319de00700 1 -- 172.21.15.86:0/3078511296 wait complete. 2024-03-22T19:10:26.506 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T19:10:26.515 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T19:10:26.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T19:10:26.875 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:27.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:27 smithi086 ceph-mon[38032]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:27.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:27 smithi086 ceph-mon[38032]: pgmap v270: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:27 smithi131 ceph-mon[78559]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:27 smithi131 ceph-mon[78559]: pgmap v270: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:27 smithi167 ceph-mon[47728]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T19:10:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:27 smithi167 ceph-mon[47728]: pgmap v270: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 -- 172.21.15.86:0/172380000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744100de0 msgr2=0x7f3744101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 --2- 172.21.15.86:0/172380000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744100de0 0x7f3744101200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f37340099c0 tx=0x7f3734009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 -- 172.21.15.86:0/172380000 shutdown_connections 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 --2- 172.21.15.86:0/172380000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744102980 0x7f3744109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 --2- 172.21.15.86:0/172380000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744100de0 0x7f3744101200 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 --2- 172.21.15.86:0/172380000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3744101fe0 0x7f3744102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.108+0000 7f3748c00700 1 -- 172.21.15.86:0/172380000 >> 172.21.15.86:0/172380000 conn(0x7f37440fc380 msgr2=0x7f37440fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.109+0000 7f3748c00700 1 -- 172.21.15.86:0/172380000 shutdown_connections 2024-03-22T19:10:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.109+0000 7f3748c00700 1 -- 172.21.15.86:0/172380000 wait complete. 2024-03-22T19:10:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 Processor -- start 2024-03-22T19:10:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 -- start start 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744100de0 0x7f3744196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3744101fe0 0x7f37441972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 0x7f374419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374419e850 con 0x7f3744102980 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374419e990 con 0x7f3744101fe0 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3748c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374419eb50 con 0x7f3744100de0 2024-03-22T19:10:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.110+0000 7f3742800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 0x7f374419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3741e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744100de0 0x7f3744196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 0x7f374419e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34172/0 (socket says 172.21.15.86:34172) 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 -- 172.21.15.86:0/3520868016 learned_addr learned my addr 172.21.15.86:0/3520868016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3741400700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3744101fe0 0x7f37441972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 -- 172.21.15.86:0/3520868016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744100de0 msgr2=0x7f3744196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744100de0 0x7f3744196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 -- 172.21.15.86:0/3520868016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3744101fe0 msgr2=0x7f37441972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3744101fe0 0x7f37441972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 -- 172.21.15.86:0/3520868016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340095f0 con 0x7f3744102980 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3741e00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744100de0 0x7f3744196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3742800700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 0x7f374419e280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f373c007ee0 tx=0x7f373c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00dd20 con 0x7f3744102980 2024-03-22T19:10:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f373c00f040 con 0x7f3744102980 2024-03-22T19:10:28.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0048a0 con 0x7f3744102980 2024-03-22T19:10:28.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.111+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3744073040 con 0x7f3744102980 2024-03-22T19:10:28.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.112+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37440734c0 con 0x7f3744102980 2024-03-22T19:10:28.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.113+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3744108260 con 0x7f3744102980 2024-03-22T19:10:28.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.114+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f373c003a00 con 0x7f3744102980 2024-03-22T19:10:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.116+0000 7f373aa00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f37280a0ba0 0x7f37280a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.116+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f373c0cb930 con 0x7f3744102980 2024-03-22T19:10:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.116+0000 7f3741e00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f37280a0ba0 0x7f37280a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.116+0000 7f3741e00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f37280a0ba0 0x7f37280a3060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3734006010 tx=0x7f3734000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:28.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.120+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f373c095190 con 0x7f3744102980 2024-03-22T19:10:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.371+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f374404ea80 con 0x7f3744102980 2024-03-22T19:10:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.371+0000 7f373aa00700 1 -- 172.21.15.86:0/3520868016 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2931 (secure 0 0 0) 0x7f373c0987b0 con 0x7f3744102980 2024-03-22T19:10:28.373 INFO:teuthology.orchestra.run.smithi086.stdout:e26 2024-03-22T19:10:28.373 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T19:10:28.373 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 26 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-03-22T19:09:13.618776+0000 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-03-22T19:09:21.111792+0000 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-03-22T19:10:28.374 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=24569,1=24557,2=24550,3=24524,4=24523} 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3] 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-03-22T19:10:28.375 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{0:24518} state up:standby-replay seq 1 addr [v2:172.21.15.167:6836/1695283976,v1:172.21.15.167:6837/1695283976] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{1:24557} state up:active seq 4 addr [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{1:24500} state up:standby-replay seq 1 addr [v2:172.21.15.131:6834/4106390239,v1:172.21.15.131:6835/4106390239] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{2:24550} state up:active seq 6 addr [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{2:24475} state up:standby-replay seq 1 addr [v2:172.21.15.167:6832/700195904,v1:172.21.15.167:6833/700195904] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{3:24524} state up:active seq 7 addr [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{3:24470} state up:standby-replay seq 1 addr [v2:172.21.15.131:6832/2878260168,v1:172.21.15.131:6833/2878260168] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{4:24523} state up:active seq 8 addr [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{4:14514} state up:standby-replay seq 1 addr [v2:172.21.15.86:6838/3022543840,v1:172.21.15.86:6839/3022543840] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:28.376 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-03-22T19:10:28.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:28.377 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14460} state up:standby seq 1 addr [v2:172.21.15.86:6834/1672168335,v1:172.21.15.86:6835/1672168335] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.377 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{-1:14472} state up:standby seq 1 addr [v2:172.21.15.86:6836/3289005201,v1:172.21.15.86:6837/3289005201] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T19:10:28.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.373+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f37280a0ba0 msgr2=0x7f37280a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:28.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.373+0000 7f3748c00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f37280a0ba0 0x7f37280a3060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3734006010 tx=0x7f3734000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.373+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 msgr2=0x7f374419e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.373+0000 7f3748c00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 0x7f374419e280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f373c007ee0 tx=0x7f373c00d3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 shutdown_connections 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f37280a0ba0 0x7f37280a3060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744102980 0x7f374419e280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3744100de0 0x7f3744196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 --2- 172.21.15.86:0/3520868016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3744101fe0 0x7f37441972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 >> 172.21.15.86:0/3520868016 conn(0x7f37440fc380 msgr2=0x7f37441053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 shutdown_connections 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:28.374+0000 7f3748c00700 1 -- 172.21.15.86:0/3520868016 wait complete. 2024-03-22T19:10:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:10:28.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-22T19:10:29.131 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:29.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:29 smithi086 ceph-mon[38032]: pgmap v271: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:29.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:29 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3520868016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T19:10:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:29 smithi167 ceph-mon[47728]: pgmap v271: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:29 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3520868016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T19:10:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:29 smithi131 ceph-mon[78559]: pgmap v271: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:10:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:29 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3520868016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 -- 172.21.15.86:0/691362320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570100de0 msgr2=0x7fc570101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 --2- 172.21.15.86:0/691362320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570100de0 0x7fc570101200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc5600099c0 tx=0x7fc560009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 -- 172.21.15.86:0/691362320 shutdown_connections 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 --2- 172.21.15.86:0/691362320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc570102980 0x7fc570109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 --2- 172.21.15.86:0/691362320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570100de0 0x7fc570101200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 --2- 172.21.15.86:0/691362320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc570102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 -- 172.21.15.86:0/691362320 >> 172.21.15.86:0/691362320 conn(0x7fc5700fc380 msgr2=0x7fc5700fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 -- 172.21.15.86:0/691362320 shutdown_connections 2024-03-22T19:10:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.255+0000 7fc576600700 1 -- 172.21.15.86:0/691362320 wait complete. 2024-03-22T19:10:30.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 Processor -- start 2024-03-22T19:10:30.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 -- start start 2024-03-22T19:10:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc570100de0 0x7fc570074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc5700731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570102980 0x7fc5700736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570073e80 con 0x7fc570101fe0 2024-03-22T19:10:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570073ff0 con 0x7fc570102980 2024-03-22T19:10:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.257+0000 7fc576600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5701a0d90 con 0x7fc570100de0 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc5700731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc5700731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51120/0 (socket says 172.21.15.86:51120) 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 -- 172.21.15.86:0/2770876033 learned_addr learned my addr 172.21.15.86:0/2770876033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56f400700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc570100de0 0x7fc570074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56fe00700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570102980 0x7fc5700736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 -- 172.21.15.86:0/2770876033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc570100de0 msgr2=0x7fc570074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc570100de0 0x7fc570074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 -- 172.21.15.86:0/2770876033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570102980 msgr2=0x7fc5700736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570102980 0x7fc5700736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 -- 172.21.15.86:0/2770876033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c009520 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc56ea00700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc5700731a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00eb30 tx=0x7fc55c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c009880 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc55c0044e0 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.258+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5600095f0 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.259+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c003d80 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.259+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5701a1350 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.260+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570108290 con 0x7fc570101fe0 2024-03-22T19:10:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.261+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc55c003f20 con 0x7fc570101fe0 2024-03-22T19:10:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.263+0000 7fc567e00700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5540a0ba0 0x7fc5540a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.263+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc55c014070 con 0x7fc570101fe0 2024-03-22T19:10:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.263+0000 7fc56f400700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5540a0ba0 0x7fc5540a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.263+0000 7fc56f400700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5540a0ba0 0x7fc5540a3060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc560003730 tx=0x7fc560000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:30.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.266+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc55c095ee0 con 0x7fc570101fe0 2024-03-22T19:10:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.470+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc57004ea80 con 0x7fc570101fe0 2024-03-22T19:10:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.471+0000 7fc567e00700 1 -- 172.21.15.86:0/2770876033 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) v1 ==== 56+0+5236 (secure 0 0 0) 0x7fc55c099500 con 0x7fc570101fe0 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 82 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:fsid b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-03-22T19:02:55.168034+0000 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-03-22T19:09:51.217885+0000 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 26 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-03-22T19:10:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-03-22T19:10:30.474 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release reef 2024-03-22T19:10:30.474 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-03-22T19:10:30.474 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-22T19:10:30.474 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-22T19:10:30.474 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-22T19:10:30.474 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/2783447556,v1:172.21.15.86:6803/2783447556] [v2:172.21.15.86:6804/2783447556,v1:172.21.15.86:6805/2783447556] exists,up edf49b32-3333-4f40-bf4d-fb6ac26e3890 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/3559403915,v1:172.21.15.131:6801/3559403915] [v2:172.21.15.131:6802/3559403915,v1:172.21.15.131:6803/3559403915] exists,up 3dc78998-5904-4af2-97fd-880603deeb09 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/2732206899,v1:172.21.15.167:6801/2732206899] [v2:172.21.15.167:6802/2732206899,v1:172.21.15.167:6803/2732206899] exists,up 84f39745-e2e8-4b26-8041-28d4e8914192 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/1218616366,v1:172.21.15.86:6811/1218616366] [v2:172.21.15.86:6812/1218616366,v1:172.21.15.86:6813/1218616366] exists,up 4c78c338-9e45-43c1-abed-bed5550c0582 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 32 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/2968759131,v1:172.21.15.131:6809/2968759131] [v2:172.21.15.131:6810/2968759131,v1:172.21.15.131:6811/2968759131] exists,up 58f48274-3606-4e03-bd1c-1f58c186882d 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 37 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/634422158,v1:172.21.15.167:6809/634422158] [v2:172.21.15.167:6810/634422158,v1:172.21.15.167:6811/634422158] exists,up 4140705c-b24d-488d-8db0-7f6093013c8a 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 44 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/4292626130,v1:172.21.15.86:6819/4292626130] [v2:172.21.15.86:6820/4292626130,v1:172.21.15.86:6821/4292626130] exists,up e9f54546-8e74-4eb8-9489-9b5155f0501e 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 49 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6816/1354625661,v1:172.21.15.131:6817/1354625661] [v2:172.21.15.131:6818/1354625661,v1:172.21.15.131:6819/1354625661] exists,up 3e490f98-55e1-44e0-8c95-234d0d2b05d5 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 56 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/2277591889,v1:172.21.15.167:6817/2277591889] [v2:172.21.15.167:6818/2277591889,v1:172.21.15.167:6819/2277591889] exists,up e3b7bdec-dde1-4853-9969-3de47c4ba84e 2024-03-22T19:10:30.475 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 63 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/1269690910,v1:172.21.15.86:6827/1269690910] [v2:172.21.15.86:6828/1269690910,v1:172.21.15.86:6829/1269690910] exists,up 285c5d25-0d34-4b60-a709-e8ba94d47a45 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 68 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6824/407606440,v1:172.21.15.131:6825/407606440] [v2:172.21.15.131:6826/407606440,v1:172.21.15.131:6827/407606440] exists,up 2c9eab90-57b4-4b6c-bed1-8f589dc8d971 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 75 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/736141441,v1:172.21.15.167:6825/736141441] [v2:172.21.15.167:6826/736141441,v1:172.21.15.167:6827/736141441] exists,up 1101cd32-ff32-47ff-8023-01e657e68b67 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.a [2,11] 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.16 [2,4] 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.21 [5,9] 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.24 [4,9] 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.36 [4,9] 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1950872310 expires 2024-03-23T19:03:50.084047+0000 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3835111951 expires 2024-03-23T19:03:25.120388+0000 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/2748771516 expires 2024-03-23T19:03:50.084047+0000 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/2290148199 expires 2024-03-23T19:03:25.120388+0000 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/2748771516 expires 2024-03-23T19:03:50.084047+0000 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2112495366 expires 2024-03-23T19:03:25.120388+0000 2024-03-22T19:10:30.476 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/764935385 expires 2024-03-23T19:03:50.084047+0000 2024-03-22T19:10:30.477 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/583320025 expires 2024-03-23T19:03:25.120388+0000 2024-03-22T19:10:30.477 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1207258255 expires 2024-03-23T19:03:50.084047+0000 2024-03-22T19:10:30.477 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/2290148199 expires 2024-03-23T19:03:25.120388+0000 2024-03-22T19:10:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.473+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5540a0ba0 msgr2=0x7fc5540a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.473+0000 7fc576600700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5540a0ba0 0x7fc5540a3060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc560003730 tx=0x7fc560000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 msgr2=0x7fc5700731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc5700731a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00eb30 tx=0x7fc55c00ee40 comp rx=0 tx=0).stop 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 shutdown_connections 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc5540a0ba0 0x7fc5540a3060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc570102980 0x7fc5700736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc570100de0 0x7fc570074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 --2- 172.21.15.86:0/2770876033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc570101fe0 0x7fc5700731a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 >> 172.21.15.86:0/2770876033 conn(0x7fc5700fc380 msgr2=0x7fc5701053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 shutdown_connections 2024-03-22T19:10:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:30.474+0000 7fc576600700 1 -- 172.21.15.86:0/2770876033 wait complete. 2024-03-22T19:10:30.901 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-22T19:10:30.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:31.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 -- 172.21.15.86:0/2400845712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e4100c50 msgr2=0x7f10e4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 --2- 172.21.15.86:0/2400845712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e4100c50 0x7f10e4101070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f10d40099c0 tx=0x7f10d4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 -- 172.21.15.86:0/2400845712 shutdown_connections 2024-03-22T19:10:31.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 --2- 172.21.15.86:0/2400845712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e41027e0 0x7f10e4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 --2- 172.21.15.86:0/2400845712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e4100c50 0x7f10e4101070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 --2- 172.21.15.86:0/2400845712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10e4101e40 0x7f10e41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 -- 172.21.15.86:0/2400845712 >> 172.21.15.86:0/2400845712 conn(0x7f10e40fc1c0 msgr2=0x7f10e40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:31.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 -- 172.21.15.86:0/2400845712 shutdown_connections 2024-03-22T19:10:31.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.107+0000 7f10ea800700 1 -- 172.21.15.86:0/2400845712 wait complete. 2024-03-22T19:10:31.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.108+0000 7f10ea800700 1 Processor -- start 2024-03-22T19:10:31.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 -- start start 2024-03-22T19:10:31.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e4100c50 0x7f10e4198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10e4101e40 0x7f10e4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 0x7f10e41a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e41a0960 con 0x7f10e41027e0 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e41a0aa0 con 0x7f10e4100c50 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10ea800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e41a0c30 con 0x7f10e4101e40 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 0x7f10e41a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e4100c50 0x7f10e4198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 0x7f10e41a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51144/0 (socket says 172.21.15.86:51144) 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 -- 172.21.15.86:0/526376820 learned_addr learned my addr 172.21.15.86:0/526376820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e2a00700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10e4101e40 0x7f10e4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 -- 172.21.15.86:0/526376820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10e4101e40 msgr2=0x7f10e4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10e4101e40 0x7f10e4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 -- 172.21.15.86:0/526376820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e4100c50 msgr2=0x7f10e4198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e4100c50 0x7f10e4198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.109+0000 7f10e3e00700 1 -- 172.21.15.86:0/526376820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40095f0 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.110+0000 7f10e3e00700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 0x7f10e41a0390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00ef30 tx=0x7f10dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.110+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc018070 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.110+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10dc00f040 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.110+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e41a0eb0 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.110+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00cd00 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.110+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e41a1330 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.111+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e40679e0 con 0x7f10e41027e0 2024-03-22T19:10:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.113+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f10dc003a00 con 0x7f10e41027e0 2024-03-22T19:10:31.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.114+0000 7f10dbe00700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10c80a0b20 0x7f10c80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.114+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f10dc0cbc80 con 0x7f10e41027e0 2024-03-22T19:10:31.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.114+0000 7f10e3400700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10c80a0b20 0x7f10c80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.115+0000 7f10e3400700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10c80a0b20 0x7f10c80a2fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f10d4003730 tx=0x7f10d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.118+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f10dc095570 con 0x7f10e41027e0 2024-03-22T19:10:31.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.360+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10e41a1690 con 0x7f10e41027e0 2024-03-22T19:10:31.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.362+0000 7f10dbe00700 1 -- 172.21.15.86:0/526376820 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7f10dc098b90 con 0x7f10e41027e0 2024-03-22T19:10:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:31.364 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10c80a0b20 msgr2=0x7f10c80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10c80a0b20 0x7f10c80a2fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f10d4003730 tx=0x7f10d4000bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 msgr2=0x7f10e41a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 0x7f10e41a0390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00ef30 tx=0x7f10dc00c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 shutdown_connections 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10c80a0b20 0x7f10c80a2fe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e41027e0 0x7f10e41a0390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e4100c50 0x7f10e4198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 --2- 172.21.15.86:0/526376820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10e4101e40 0x7f10e4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.364+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 >> 172.21.15.86:0/526376820 conn(0x7f10e40fc1c0 msgr2=0x7f10e4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:31.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.365+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 shutdown_connections 2024-03-22T19:10:31.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.365+0000 7f10ea800700 1 -- 172.21.15.86:0/526376820 wait complete. 2024-03-22T19:10:31.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:31.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:31.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:31 smithi086 ceph-mon[38032]: pgmap v272: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:31.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:31 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2770876033' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-22T19:10:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:31 smithi167 ceph-mon[47728]: pgmap v272: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:31 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2770876033' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-22T19:10:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 -- 172.21.15.86:0/2331672587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec101d80 msgr2=0x7fdaec1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 --2- 172.21.15.86:0/2331672587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec101d80 0x7fdaec1021e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdad80099c0 tx=0x7fdad8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 -- 172.21.15.86:0/2331672587 shutdown_connections 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 --2- 172.21.15.86:0/2331672587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaec102720 0x7fdaec108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 --2- 172.21.15.86:0/2331672587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec100b90 0x7fdaec100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 --2- 172.21.15.86:0/2331672587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec101d80 0x7fdaec1021e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 -- 172.21.15.86:0/2331672587 >> 172.21.15.86:0/2331672587 conn(0x7fdaec0fc0e0 msgr2=0x7fdaec0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 -- 172.21.15.86:0/2331672587 shutdown_connections 2024-03-22T19:10:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.579+0000 7fdaebe00700 1 -- 172.21.15.86:0/2331672587 wait complete. 2024-03-22T19:10:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.580+0000 7fdaebe00700 1 Processor -- start 2024-03-22T19:10:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.580+0000 7fdaebe00700 1 -- start start 2024-03-22T19:10:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 0x7fdaec198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaebe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaec101d80 0x7fdaec199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaebe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec102720 0x7fdaec1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaebe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec1a0840 con 0x7fdaec100b90 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaebe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec1a09b0 con 0x7fdaec102720 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaebe00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec1a0b70 con 0x7fdaec101d80 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 0x7fdaec198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 0x7fdaec198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51148/0 (socket says 172.21.15.86:51148) 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 -- 172.21.15.86:0/41622126 learned_addr learned my addr 172.21.15.86:0/41622126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeb400700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec102720 0x7fdaec1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaea000700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaec101d80 0x7fdaec199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 -- 172.21.15.86:0/41622126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaec101d80 msgr2=0x7fdaec199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaec101d80 0x7fdaec199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 -- 172.21.15.86:0/41622126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec102720 msgr2=0x7fdaec1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec102720 0x7fdaec1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 -- 172.21.15.86:0/41622126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad80095f0 con 0x7fdaec100b90 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.581+0000 7fdaeaa00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 0x7fdaec198d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00e970 tx=0x7fdadc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.582+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc009790 con 0x7fdaec100b90 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.582+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdadc00eec0 con 0x7fdaec100b90 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.582+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec1a0e50 con 0x7fdaec100b90 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.582+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc003cf0 con 0x7fdaec100b90 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.582+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec1a1280 con 0x7fdaec100b90 2024-03-22T19:10:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.583+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec067970 con 0x7fdaec100b90 2024-03-22T19:10:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.585+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fdadc00ccd0 con 0x7fdaec100b90 2024-03-22T19:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.586+0000 7fdae3400700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdad00a0ba0 0x7fdad00a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.586+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fdadc014070 con 0x7fdaec100b90 2024-03-22T19:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.586+0000 7fdaea000700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdad00a0ba0 0x7fdad00a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.587+0000 7fdaea000700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdad00a0ba0 0x7fdad00a3060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdad800b580 tx=0x7fdad801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:31.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.590+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdadc095c10 con 0x7fdaec100b90 2024-03-22T19:10:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:31 smithi131 ceph-mon[78559]: pgmap v272: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:31 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2770876033' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-22T19:10:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.794+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdaec1a1500 con 0x7fdaec100b90 2024-03-22T19:10:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.794+0000 7fdae3400700 1 -- 172.21.15.86:0/41622126 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdadc099230 con 0x7fdaec100b90 2024-03-22T19:10:31.796 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdad00a0ba0 msgr2=0x7fdad00a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdad00a0ba0 0x7fdad00a3060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdad800b580 tx=0x7fdad801a040 comp rx=0 tx=0).stop 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 msgr2=0x7fdaec198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 0x7fdaec198d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00e970 tx=0x7fdadc00ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 shutdown_connections 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdad00a0ba0 0x7fdad00a3060 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec102720 0x7fdaec1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaec100b90 0x7fdaec198d50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 --2- 172.21.15.86:0/41622126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaec101d80 0x7fdaec199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 >> 172.21.15.86:0/41622126 conn(0x7fdaec0fc0e0 msgr2=0x7fdaec107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 shutdown_connections 2024-03-22T19:10:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:31.797+0000 7fdaebe00700 1 -- 172.21.15.86:0/41622126 wait complete. 2024-03-22T19:10:31.817 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 -- 172.21.15.86:0/2767633739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 msgr2=0x7f549010b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 --2- 172.21.15.86:0/2767633739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f549010b090 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f548800b0a0 tx=0x7f548800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 -- 172.21.15.86:0/2767633739 shutdown_connections 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 --2- 172.21.15.86:0/2767633739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f549010b090 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 --2- 172.21.15.86:0/2767633739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5490074d90 0x7f54900731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 --2- 172.21.15.86:0/2767633739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54900737b0 0x7f5490073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 -- 172.21.15.86:0/2767633739 >> 172.21.15.86:0/2767633739 conn(0x7f54900fc2c0 msgr2=0x7f54900fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 -- 172.21.15.86:0/2767633739 shutdown_connections 2024-03-22T19:10:32.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.011+0000 7f5497c00700 1 -- 172.21.15.86:0/2767633739 wait complete. 2024-03-22T19:10:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.012+0000 7f5497c00700 1 Processor -- start 2024-03-22T19:10:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.012+0000 7f5497c00700 1 -- start start 2024-03-22T19:10:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5497c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54900737b0 0x7f5490198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5497c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5490074d90 0x7f5490199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5497c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f54901a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5497c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901a0ac0 con 0x7f5490108c20 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5497c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901a0c00 con 0x7f54900737b0 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5497c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901a0d90 con 0x7f5490074d90 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f54901a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f54901a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51172/0 (socket says 172.21.15.86:51172) 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495c00700 1 -- 172.21.15.86:0/3834766370 learned_addr learned my addr 172.21.15.86:0/3834766370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f548fe00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5490074d90 0x7f5490199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495200700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54900737b0 0x7f5490198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495c00700 1 -- 172.21.15.86:0/3834766370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5490074d90 msgr2=0x7f5490199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5490074d90 0x7f5490199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.013+0000 7f5495c00700 1 -- 172.21.15.86:0/3834766370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54900737b0 msgr2=0x7f5490198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f5495c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54900737b0 0x7f5490198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f5495c00700 1 -- 172.21.15.86:0/3834766370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009d00 con 0x7f5490108c20 2024-03-22T19:10:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f5495c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f54901a04f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5488007a60 tx=0x7f5488007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800e040 con 0x7f5490108c20 2024-03-22T19:10:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54901a1010 con 0x7f5490108c20 2024-03-22T19:10:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54901a1430 con 0x7f5490108c20 2024-03-22T19:10:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5488009b80 con 0x7f5490108c20 2024-03-22T19:10:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.014+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548801b460 con 0x7f5490108c20 2024-03-22T19:10:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.015+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470005320 con 0x7f5490108c20 2024-03-22T19:10:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.017+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f5488004c50 con 0x7f5490108c20 2024-03-22T19:10:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.018+0000 7f548d600700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f54780a0ba0 0x7f54780a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.019+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f54880cd670 con 0x7f5490108c20 2024-03-22T19:10:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.019+0000 7f5495200700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f54780a0ba0 0x7f54780a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.019+0000 7f5495200700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f54780a0ba0 0x7f54780a3060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5484009c60 tx=0x7f5484009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.020+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5488096f60 con 0x7f5490108c20 2024-03-22T19:10:32.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.231+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5470005f70 con 0x7f5490108c20 2024-03-22T19:10:32.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.232+0000 7f548d600700 1 -- 172.21.15.86:0/3834766370 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f548809a580 con 0x7f5490108c20 2024-03-22T19:10:32.234 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:32.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f54780a0ba0 msgr2=0x7f54780a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f54780a0ba0 0x7f54780a3060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5484009c60 tx=0x7f5484009260 comp rx=0 tx=0).stop 2024-03-22T19:10:32.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 msgr2=0x7f54901a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f54901a04f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5488007a60 tx=0x7f5488007930 comp rx=0 tx=0).stop 2024-03-22T19:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 shutdown_connections 2024-03-22T19:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f54780a0ba0 0x7f54780a3060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5490108c20 0x7f54901a04f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54900737b0 0x7f5490198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 --2- 172.21.15.86:0/3834766370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5490074d90 0x7f5490199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.234+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 >> 172.21.15.86:0/3834766370 conn(0x7f54900fc2c0 msgr2=0x7f5490104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:32.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.235+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 shutdown_connections 2024-03-22T19:10:32.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.235+0000 7f5497c00700 1 -- 172.21.15.86:0/3834766370 wait complete. 2024-03-22T19:10:32.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 -- 172.21.15.86:0/2978711713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 msgr2=0x7f96a8101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 --2- 172.21.15.86:0/2978711713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8101160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f969c009920 tx=0x7f969c009c30 comp rx=0 tx=0).stop 2024-03-22T19:10:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 -- 172.21.15.86:0/2978711713 shutdown_connections 2024-03-22T19:10:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 --2- 172.21.15.86:0/2978711713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 0x7f96a8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 --2- 172.21.15.86:0/2978711713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8101160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 --2- 172.21.15.86:0/2978711713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a8101f30 0x7f96a8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 -- 172.21.15.86:0/2978711713 >> 172.21.15.86:0/2978711713 conn(0x7f96a80fc2b0 msgr2=0x7f96a80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:32.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 -- 172.21.15.86:0/2978711713 shutdown_connections 2024-03-22T19:10:32.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.448+0000 7f96afe00700 1 -- 172.21.15.86:0/2978711713 wait complete. 2024-03-22T19:10:32.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.449+0000 7f96afe00700 1 Processor -- start 2024-03-22T19:10:32.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.449+0000 7f96afe00700 1 -- start start 2024-03-22T19:10:32.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a8101f30 0x7f96a80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96afe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 0x7f96a80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96afe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8073e80 con 0x7f96a8101f30 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96afe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8073ff0 con 0x7f96a8100d40 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96afe00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81a0e30 con 0x7f96a81028d0 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:38930/0 (socket says 172.21.15.86:38930) 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 -- 172.21.15.86:0/1154914978 learned_addr learned my addr 172.21.15.86:0/1154914978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96aca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a8101f30 0x7f96a80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ade00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 0x7f96a80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 -- 172.21.15.86:0/1154914978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 msgr2=0x7f96a80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 0x7f96a80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 -- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a8101f30 msgr2=0x7f96a80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a8101f30 0x7f96a80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ad400700 1 -- 172.21.15.86:0/1154914978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940095f0 con 0x7f96a8100d40 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.450+0000 7f96ade00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 0x7f96a80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.451+0000 7f96ad400700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8074b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f969c009230 tx=0x7f969c00bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.451+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c01c070 con 0x7f96a8100d40 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.451+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c00bba0 con 0x7f96a8100d40 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.451+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0049c0 con 0x7f96a8100d40 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.451+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c009520 con 0x7f96a8100d40 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.451+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a81a1240 con 0x7f96a8100d40 2024-03-22T19:10:32.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.452+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a81082b0 con 0x7f96a8100d40 2024-03-22T19:10:32.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.454+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f969c020070 con 0x7f96a8100d40 2024-03-22T19:10:32.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.455+0000 7f96a6000700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f96900a0b20 0x7f96900a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.455+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f969c0cd550 con 0x7f96a8100d40 2024-03-22T19:10:32.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.455+0000 7f96aca00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f96900a0b20 0x7f96900a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.456+0000 7f96aca00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f96900a0b20 0x7f96900a2fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9694005fd0 tx=0x7f9694009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:32.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.457+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f969c096e40 con 0x7f96a8100d40 2024-03-22T19:10:32.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:32 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/526376820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:32.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:32 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/41622126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:32.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:32 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/526376820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:32.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:32 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/41622126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.665+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96a802d0f0 con 0x7f96a8100d40 2024-03-22T19:10:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.666+0000 7f96a6000700 1 -- 172.21.15.86:0/1154914978 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f969c09a460 con 0x7f96a8100d40 2024-03-22T19:10:32.667 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f96900a0b20 msgr2=0x7f96900a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f96900a0b20 0x7f96900a2fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9694005fd0 tx=0x7f9694009310 comp rx=0 tx=0).stop 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 msgr2=0x7f96a8074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8074b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f969c009230 tx=0x7f969c00bef0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 shutdown_connections 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f96900a0b20 0x7f96900a2fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a81028d0 0x7f96a80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a8100d40 0x7f96a8074b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 --2- 172.21.15.86:0/1154914978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a8101f30 0x7f96a80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 >> 172.21.15.86:0/1154914978 conn(0x7f96a80fc2b0 msgr2=0x7f96a81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 shutdown_connections 2024-03-22T19:10:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.668+0000 7f96afe00700 1 -- 172.21.15.86:0/1154914978 wait complete. 2024-03-22T19:10:32.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:32.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:32 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/526376820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:32.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:32 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/41622126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 -- 172.21.15.86:0/3868353439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4073810 msgr2=0x7f43d4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 --2- 172.21.15.86:0/3868353439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4073810 0x7f43d4073c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009990 tx=0x7f43c0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 -- 172.21.15.86:0/3868353439 shutdown_connections 2024-03-22T19:10:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 --2- 172.21.15.86:0/3868353439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4108b10 0x7f43d410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 --2- 172.21.15.86:0/3868353439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d4074df0 0x7f43d4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 --2- 172.21.15.86:0/3868353439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4073810 0x7f43d4073c90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 -- 172.21.15.86:0/3868353439 >> 172.21.15.86:0/3868353439 conn(0x7f43d40fc1c0 msgr2=0x7f43d40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 -- 172.21.15.86:0/3868353439 shutdown_connections 2024-03-22T19:10:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.877+0000 7f43db600700 1 -- 172.21.15.86:0/3868353439 wait complete. 2024-03-22T19:10:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.879+0000 7f43db600700 1 Processor -- start 2024-03-22T19:10:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.879+0000 7f43db600700 1 -- start start 2024-03-22T19:10:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.879+0000 7f43db600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4073810 0x7f43d4198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.879+0000 7f43db600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d4074df0 0x7f43d4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43db600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 0x7f43d41a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43db600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a0960 con 0x7f43d4108b10 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43db600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a0aa0 con 0x7f43d4073810 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43db600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a0c30 con 0x7f43d4074df0 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4073810 0x7f43d4198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 0x7f43d41a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d4074df0 0x7f43d4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 0x7f43d41a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51202/0 (socket says 172.21.15.86:51202) 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4073810 0x7f43d4198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:38954/0 (socket says 172.21.15.86:38954) 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 -- 172.21.15.86:0/82432677 learned_addr learned my addr 172.21.15.86:0/82432677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 -- 172.21.15.86:0/82432677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d4074df0 msgr2=0x7f43d4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d4074df0 0x7f43d4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 -- 172.21.15.86:0/82432677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4073810 msgr2=0x7f43d4198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:32.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4073810 0x7f43d4198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:32.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 -- 172.21.15.86:0/82432677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c00095f0 con 0x7f43d4108b10 2024-03-22T19:10:32.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.880+0000 7f43d9600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 0x7f43d41a0390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00ef30 tx=0x7f43cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.881+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc018070 con 0x7f43d4108b10 2024-03-22T19:10:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.881+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43cc00f040 con 0x7f43d4108b10 2024-03-22T19:10:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.881+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d41a0eb0 con 0x7f43d4108b10 2024-03-22T19:10:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.881+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00cd00 con 0x7f43d4108b10 2024-03-22T19:10:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.881+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41a1380 con 0x7f43d4108b10 2024-03-22T19:10:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.882+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d41078d0 con 0x7f43d4108b10 2024-03-22T19:10:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.884+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f43cc003a00 con 0x7f43d4108b10 2024-03-22T19:10:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.885+0000 7f43d1600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f43bc0a0ba0 0x7f43bc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.885+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f43cc0cbc50 con 0x7f43d4108b10 2024-03-22T19:10:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.885+0000 7f43d8c00700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f43bc0a0ba0 0x7f43bc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.886+0000 7f43d8c00700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f43bc0a0ba0 0x7f43bc0a3060 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009c60 tx=0x7f43c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:32.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:32.889+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f43cc095540 con 0x7f43d4108b10 2024-03-22T19:10:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.143+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43d40679e0 con 0x7f43d4108b10 2024-03-22T19:10:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.144+0000 7f43d1600700 1 -- 172.21.15.86:0/82432677 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f43cc098b60 con 0x7f43d4108b10 2024-03-22T19:10:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:33.146 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:09:21.111792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.146+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f43bc0a0ba0 msgr2=0x7f43bc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.146+0000 7f43db600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f43bc0a0ba0 0x7f43bc0a3060 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009c60 tx=0x7f43c8009260 comp rx=0 tx=0).stop 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.146+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 msgr2=0x7f43d41a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.146+0000 7f43db600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 0x7f43d41a0390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00ef30 tx=0x7f43cc00c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 shutdown_connections 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f43bc0a0ba0 0x7f43bc0a3060 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d4108b10 0x7f43d41a0390 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4073810 0x7f43d4198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 --2- 172.21.15.86:0/82432677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d4074df0 0x7f43d4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 >> 172.21.15.86:0/82432677 conn(0x7f43d40fc1c0 msgr2=0x7f43d4104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 shutdown_connections 2024-03-22T19:10:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.147+0000 7f43db600700 1 -- 172.21.15.86:0/82432677 wait complete. 2024-03-22T19:10:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:10:33.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:33.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 -- 172.21.15.86:0/2126573331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760108b10 msgr2=0x7fa76010afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 --2- 172.21.15.86:0/2126573331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760108b10 0x7fa76010afa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa75800b0a0 tx=0x7fa75800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 -- 172.21.15.86:0/2126573331 shutdown_connections 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 --2- 172.21.15.86:0/2126573331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760108b10 0x7fa76010afa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 --2- 172.21.15.86:0/2126573331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa760074df0 0x7fa760073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 --2- 172.21.15.86:0/2126573331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760073810 0x7fa760073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 -- 172.21.15.86:0/2126573331 >> 172.21.15.86:0/2126573331 conn(0x7fa7600fc1c0 msgr2=0x7fa7600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 -- 172.21.15.86:0/2126573331 shutdown_connections 2024-03-22T19:10:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.360+0000 7fa765200700 1 -- 172.21.15.86:0/2126573331 wait complete. 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 Processor -- start 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 -- start start 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa760073810 0x7fa760198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760074df0 0x7fa760199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 0x7fa7601a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601a0a00 con 0x7fa760108b10 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601a0b40 con 0x7fa760074df0 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa765200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601a0cd0 con 0x7fa760073810 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.361+0000 7fa75e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 0x7fa7601a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 0x7fa7601a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51218/0 (socket says 172.21.15.86:51218) 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 -- 172.21.15.86:0/1386694713 learned_addr learned my addr 172.21.15.86:0/1386694713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75de00700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa760073810 0x7fa760198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75d400700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760074df0 0x7fa760199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 -- 172.21.15.86:0/1386694713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa760073810 msgr2=0x7fa760198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa760073810 0x7fa760198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 -- 172.21.15.86:0/1386694713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760074df0 msgr2=0x7fa760199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760074df0 0x7fa760199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 -- 172.21.15.86:0/1386694713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa758009d00 con 0x7fa760108b10 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75d400700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760074df0 0x7fa760199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa75e800700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 0x7fa7601a0430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa758007a60 tx=0x7fa758007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75800e040 con 0x7fa760108b10 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa758004c50 con 0x7fa760108b10 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580052a0 con 0x7fa760108b10 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.362+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7601a0f50 con 0x7fa760108b10 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.363+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7601a1320 con 0x7fa760108b10 2024-03-22T19:10:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.364+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7601078d0 con 0x7fa760108b10 2024-03-22T19:10:33.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.365+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fa758005400 con 0x7fa760108b10 2024-03-22T19:10:33.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.367+0000 7fa756a00700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7440a0b20 0x7fa7440a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.368+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa7580cd8d0 con 0x7fa760108b10 2024-03-22T19:10:33.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.368+0000 7fa75de00700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7440a0b20 0x7fa7440a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.368+0000 7fa75de00700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7440a0b20 0x7fa7440a2fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa750009c60 tx=0x7fa750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:33.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.372+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa7580971c0 con 0x7fa760108b10 2024-03-22T19:10:33.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:33 smithi086 ceph-mon[38032]: pgmap v273: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:33.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3834766370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:33.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1154914978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:33.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/82432677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:33 smithi167 ceph-mon[47728]: pgmap v273: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3834766370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1154914978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/82432677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.613+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa76002ccb0 con 0x7fa760108b10 2024-03-22T19:10:33.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.614+0000 7fa756a00700 1 -- 172.21.15.86:0/1386694713 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7fa75809a7e0 con 0x7fa760108b10 2024-03-22T19:10:33.616 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:33.616 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7440a0b20 msgr2=0x7fa7440a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7440a0b20 0x7fa7440a2fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa750009c60 tx=0x7fa750009260 comp rx=0 tx=0).stop 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 msgr2=0x7fa7601a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 0x7fa7601a0430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa758007a60 tx=0x7fa758007930 comp rx=0 tx=0).stop 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 shutdown_connections 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7440a0b20 0x7fa7440a2fe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa760108b10 0x7fa7601a0430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa760073810 0x7fa760198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 --2- 172.21.15.86:0/1386694713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa760074df0 0x7fa760199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 >> 172.21.15.86:0/1386694713 conn(0x7fa7600fc1c0 msgr2=0x7fa760104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 shutdown_connections 2024-03-22T19:10:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.617+0000 7fa765200700 1 -- 172.21.15.86:0/1386694713 wait complete. 2024-03-22T19:10:33.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:33.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:33 smithi131 ceph-mon[78559]: pgmap v273: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:10:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3834766370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1154914978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/82432677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:33.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.830+0000 7f7086200700 1 -- 172.21.15.86:0/996299158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7080101f30 msgr2=0x7f7080102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.830+0000 7f7086200700 1 --2- 172.21.15.86:0/996299158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7080101f30 0x7f7080102390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f706c0099c0 tx=0x7f706c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 -- 172.21.15.86:0/996299158 shutdown_connections 2024-03-22T19:10:33.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 --2- 172.21.15.86:0/996299158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70801028d0 0x7f7080109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 --2- 172.21.15.86:0/996299158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 0x7f7080101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 --2- 172.21.15.86:0/996299158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7080101f30 0x7f7080102390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 -- 172.21.15.86:0/996299158 >> 172.21.15.86:0/996299158 conn(0x7f70800fc2b0 msgr2=0x7f70800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:33.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 -- 172.21.15.86:0/996299158 shutdown_connections 2024-03-22T19:10:33.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.831+0000 7f7086200700 1 -- 172.21.15.86:0/996299158 wait complete. 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 Processor -- start 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 -- start start 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 0x7f7080198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7080101f30 0x7f70801993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801028d0 0x7f70801a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801a09e0 con 0x7f70801028d0 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801a0b20 con 0x7f7080100d40 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f7086200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801a0cb0 con 0x7f7080101f30 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f707ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7080101f30 0x7f70801993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f707fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801028d0 0x7f70801a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f707ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7080101f30 0x7f70801993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:58598/0 (socket says 172.21.15.86:58598) 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f707ea00700 1 -- 172.21.15.86:0/1657899630 learned_addr learned my addr 172.21.15.86:0/1657899630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.832+0000 7f707f400700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 0x7f7080198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707fe00700 1 -- 172.21.15.86:0/1657899630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7080101f30 msgr2=0x7f70801993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707fe00700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7080101f30 0x7f70801993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707fe00700 1 -- 172.21.15.86:0/1657899630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 msgr2=0x7f7080198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707fe00700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 0x7f7080198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707fe00700 1 -- 172.21.15.86:0/1657899630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0095f0 con 0x7f70801028d0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707f400700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 0x7f7080198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f707fe00700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801028d0 0x7f70801a0410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7078007f10 tx=0x7f707800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800ddd0 con 0x7f70801028d0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f707800f040 con 0x7f70801028d0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078005a90 con 0x7f70801028d0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70801a0f90 con 0x7f70801028d0 2024-03-22T19:10:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.833+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70801a13c0 con 0x7f70801028d0 2024-03-22T19:10:33.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.834+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7080067ac0 con 0x7f70801028d0 2024-03-22T19:10:33.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.836+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f7078004ad0 con 0x7f70801028d0 2024-03-22T19:10:33.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.837+0000 7f7077e00700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70640a0ba0 0x7f70640a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.838+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f70780cba10 con 0x7f70801028d0 2024-03-22T19:10:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.838+0000 7f707f400700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70640a0ba0 0x7f70640a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.838+0000 7f707f400700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70640a0ba0 0x7f70640a3060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7080101d90 tx=0x7f7070009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:33.841+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7078095270 con 0x7f70801028d0 2024-03-22T19:10:34.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.046+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70801a16a0 con 0x7f70801028d0 2024-03-22T19:10:34.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.047+0000 7f7077e00700 1 -- 172.21.15.86:0/1657899630 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7078098890 con 0x7f70801028d0 2024-03-22T19:10:34.049 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70640a0ba0 msgr2=0x7f70640a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70640a0ba0 0x7f70640a3060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7080101d90 tx=0x7f7070009260 comp rx=0 tx=0).stop 2024-03-22T19:10:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801028d0 msgr2=0x7f70801a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801028d0 0x7f70801a0410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7078007f10 tx=0x7f707800d3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 shutdown_connections 2024-03-22T19:10:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f70640a0ba0 0x7f70640a3060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801028d0 0x7f70801a0410 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7080100d40 0x7f7080198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 --2- 172.21.15.86:0/1657899630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7080101f30 0x7f70801993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.049+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 >> 172.21.15.86:0/1657899630 conn(0x7f70800fc2b0 msgr2=0x7f7080072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.050+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 shutdown_connections 2024-03-22T19:10:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.050+0000 7f7086200700 1 -- 172.21.15.86:0/1657899630 wait complete. 2024-03-22T19:10:34.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 -- 172.21.15.86:0/2280390276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01801040f0 msgr2=0x7f0180104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 --2- 172.21.15.86:0/2280390276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01801040f0 0x7f0180104570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f016c009990 tx=0x7f016c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 -- 172.21.15.86:0/2280390276 shutdown_connections 2024-03-22T19:10:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 --2- 172.21.15.86:0/2280390276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180104ab0 0x7f018010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 --2- 172.21.15.86:0/2280390276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180102f20 0x7f0180103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 --2- 172.21.15.86:0/2280390276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01801040f0 0x7f0180104570 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 -- 172.21.15.86:0/2280390276 >> 172.21.15.86:0/2280390276 conn(0x7f01800fe500 msgr2=0x7f0180100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 -- 172.21.15.86:0/2280390276 shutdown_connections 2024-03-22T19:10:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.266+0000 7f0184c00700 1 -- 172.21.15.86:0/2280390276 wait complete. 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 Processor -- start 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 -- start start 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180102f20 0x7f0180198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01801040f0 0x7f0180199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 0x7f01801a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01801a0a30 con 0x7f0180104ab0 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01801a0b70 con 0x7f0180102f20 2024-03-22T19:10:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.268+0000 7f0184c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01801a0d00 con 0x7f01801040f0 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 0x7f01801a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 0x7f01801a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51246/0 (socket says 172.21.15.86:51246) 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180102f20 0x7f0180198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 -- 172.21.15.86:0/3662865125 learned_addr learned my addr 172.21.15.86:0/3662865125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017d400700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01801040f0 0x7f0180199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 -- 172.21.15.86:0/3662865125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01801040f0 msgr2=0x7f0180199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01801040f0 0x7f0180199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 -- 172.21.15.86:0/3662865125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180102f20 msgr2=0x7f0180198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180102f20 0x7f0180198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 -- 172.21.15.86:0/3662865125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0095f0 con 0x7f0180104ab0 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017de00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180102f20 0x7f0180198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.269+0000 7f017e800700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 0x7f01801a0460 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f017800ef30 tx=0x7f017800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.270+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178018070 con 0x7f0180104ab0 2024-03-22T19:10:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.270+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017800f040 con 0x7f0180104ab0 2024-03-22T19:10:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.270+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01801a0f80 con 0x7f0180104ab0 2024-03-22T19:10:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.270+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017800cd00 con 0x7f0180104ab0 2024-03-22T19:10:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.270+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01801a1400 con 0x7f0180104ab0 2024-03-22T19:10:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.271+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018010a460 con 0x7f0180104ab0 2024-03-22T19:10:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.273+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f0178003a00 con 0x7f0180104ab0 2024-03-22T19:10:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.274+0000 7f0176a00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f01640a0b20 0x7f01640a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.274+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f01780cbcd0 con 0x7f0180104ab0 2024-03-22T19:10:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.274+0000 7f017de00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f01640a0b20 0x7f01640a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.275+0000 7f017de00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f01640a0b20 0x7f01640a2fe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0180102c40 tx=0x7f0170009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.278+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01780955c0 con 0x7f0180104ab0 2024-03-22T19:10:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.482+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f018002cdc0 con 0x7f0180104ab0 2024-03-22T19:10:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.482+0000 7f0176a00700 1 -- 172.21.15.86:0/3662865125 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0178098be0 con 0x7f0180104ab0 2024-03-22T19:10:34.484 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f01640a0b20 msgr2=0x7f01640a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f01640a0b20 0x7f01640a2fe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0180102c40 tx=0x7f0170009260 comp rx=0 tx=0).stop 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 msgr2=0x7f01801a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 0x7f01801a0460 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f017800ef30 tx=0x7f017800c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 shutdown_connections 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f01640a0b20 0x7f01640a2fe0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180104ab0 0x7f01801a0460 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0180102f20 0x7f0180198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.484+0000 7f0184c00700 1 --2- 172.21.15.86:0/3662865125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01801040f0 0x7f0180199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.485+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 >> 172.21.15.86:0/3662865125 conn(0x7f01800fe500 msgr2=0x7f01801075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.485+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 shutdown_connections 2024-03-22T19:10:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.485+0000 7f0184c00700 1 -- 172.21.15.86:0/3662865125 wait complete. 2024-03-22T19:10:34.502 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:34 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1386694713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:34.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:34 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1657899630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:34.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:34.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:34 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1386694713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:34.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:34 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1657899630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.697+0000 7fbb82400700 1 -- 172.21.15.86:0/1073581110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c073810 msgr2=0x7fbb7c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.697+0000 7fbb82400700 1 --2- 172.21.15.86:0/1073581110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c073810 0x7fbb7c073c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009990 tx=0x7fbb68009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 -- 172.21.15.86:0/1073581110 shutdown_connections 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 --2- 172.21.15.86:0/1073581110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c10d310 0x7fbb7c10f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 --2- 172.21.15.86:0/1073581110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7c074df0 0x7fbb7c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 --2- 172.21.15.86:0/1073581110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c073810 0x7fbb7c073c90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 -- 172.21.15.86:0/1073581110 >> 172.21.15.86:0/1073581110 conn(0x7fbb7c0fc2b0 msgr2=0x7fbb7c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 -- 172.21.15.86:0/1073581110 shutdown_connections 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.698+0000 7fbb82400700 1 -- 172.21.15.86:0/1073581110 wait complete. 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.699+0000 7fbb82400700 1 Processor -- start 2024-03-22T19:10:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.699+0000 7fbb82400700 1 -- start start 2024-03-22T19:10:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb82400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7c073810 0x7fbb7c19d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb82400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c074df0 0x7fbb7c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb82400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 0x7fbb7c1a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb82400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c1a4e70 con 0x7fbb7c10d310 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb82400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c1a4fb0 con 0x7fbb7c074df0 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb82400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c1a5140 con 0x7fbb7c073810 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 0x7fbb7c1a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c074df0 0x7fbb7c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7c073810 0x7fbb7c19d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c074df0 0x7fbb7c19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39008/0 (socket says 172.21.15.86:39008) 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 0x7fbb7c1a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51254/0 (socket says 172.21.15.86:51254) 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7aa00700 1 -- 172.21.15.86:0/3653261000 learned_addr learned my addr 172.21.15.86:0/3653261000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 -- 172.21.15.86:0/3653261000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7c073810 msgr2=0x7fbb7c19d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7c073810 0x7fbb7c19d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 -- 172.21.15.86:0/3653261000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c074df0 msgr2=0x7fbb7c19d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c074df0 0x7fbb7c19d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.700+0000 7fbb7be00700 1 -- 172.21.15.86:0/3653261000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680095f0 con 0x7fbb7c10d310 2024-03-22T19:10:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.701+0000 7fbb7be00700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 0x7fbb7c1a48a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400efc0 tx=0x7fbb7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.701+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74018070 con 0x7fbb7c10d310 2024-03-22T19:10:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.701+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c1a53c0 con 0x7fbb7c10d310 2024-03-22T19:10:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.701+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7400f040 con 0x7fbb7c10d310 2024-03-22T19:10:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.701+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400cd40 con 0x7fbb7c10d310 2024-03-22T19:10:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.701+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c1a58d0 con 0x7fbb7c10d310 2024-03-22T19:10:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.702+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c103570 con 0x7fbb7c10d310 2024-03-22T19:10:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.704+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fbb74003a00 con 0x7fbb7c10d310 2024-03-22T19:10:34.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.706+0000 7fbb73e00700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbb600a0b20 0x7fbb600a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:34.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.706+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fbb740cc720 con 0x7fbb7c10d310 2024-03-22T19:10:34.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.706+0000 7fbb7b400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbb600a0b20 0x7fbb600a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:34.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.707+0000 7fbb7b400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbb600a0b20 0x7fbb600a2fe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009c60 tx=0x7fbb6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:34.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.709+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbb74096010 con 0x7fbb7c10d310 2024-03-22T19:10:34.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:34 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1386694713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:34.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:34 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1657899630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.914+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb7c02ceb0 con 0x7fbb7c10d310 2024-03-22T19:10:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.915+0000 7fbb73e00700 1 -- 172.21.15.86:0/3653261000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb74099630 con 0x7fbb7c10d310 2024-03-22T19:10:34.916 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbb600a0b20 msgr2=0x7fbb600a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbb600a0b20 0x7fbb600a2fe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009c60 tx=0x7fbb6c009260 comp rx=0 tx=0).stop 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 msgr2=0x7fbb7c1a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 0x7fbb7c1a48a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400efc0 tx=0x7fbb7400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 shutdown_connections 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbb600a0b20 0x7fbb600a2fe0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c10d310 0x7fbb7c1a48a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.917+0000 7fbb82400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7c073810 0x7fbb7c19d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.918+0000 7fbb82400700 1 --2- 172.21.15.86:0/3653261000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c074df0 0x7fbb7c19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:34.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.918+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 >> 172.21.15.86:0/3653261000 conn(0x7fbb7c0fc2b0 msgr2=0x7fbb7c1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:34.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.918+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 shutdown_connections 2024-03-22T19:10:34.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:34.918+0000 7fbb82400700 1 -- 172.21.15.86:0/3653261000 wait complete. 2024-03-22T19:10:34.937 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 -- 172.21.15.86:0/730181343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354101e40 msgr2=0x7f93541022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 --2- 172.21.15.86:0/730181343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354101e40 0x7f93541022a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93400099c0 tx=0x7f9340009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 -- 172.21.15.86:0/730181343 shutdown_connections 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 --2- 172.21.15.86:0/730181343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93541027e0 0x7f9354109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 --2- 172.21.15.86:0/730181343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9354100c50 0x7f9354101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 --2- 172.21.15.86:0/730181343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354101e40 0x7f93541022a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 -- 172.21.15.86:0/730181343 >> 172.21.15.86:0/730181343 conn(0x7f93540fc1c0 msgr2=0x7f93540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 -- 172.21.15.86:0/730181343 shutdown_connections 2024-03-22T19:10:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.127+0000 7f935c000700 1 -- 172.21.15.86:0/730181343 wait complete. 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 Processor -- start 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 -- start start 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 0x7f9354196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9354101e40 0x7f93541970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93541027e0 0x7f935419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419e6c0 con 0x7f9354100c50 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419e800 con 0x7f93541027e0 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f935c000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419e990 con 0x7f9354101e40 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f9359600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 0x7f9354196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f9359600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 0x7f9354196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51272/0 (socket says 172.21.15.86:51272) 2024-03-22T19:10:35.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.128+0000 7f9359600700 1 -- 172.21.15.86:0/3493222599 learned_addr learned my addr 172.21.15.86:0/3493222599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9358c00700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9354101e40 0x7f93541970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f935a000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93541027e0 0x7f935419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9359600700 1 -- 172.21.15.86:0/3493222599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9354101e40 msgr2=0x7f93541970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9359600700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9354101e40 0x7f93541970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9359600700 1 -- 172.21.15.86:0/3493222599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93541027e0 msgr2=0x7f935419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9359600700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93541027e0 0x7f935419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9359600700 1 -- 172.21.15.86:0/3493222599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400095f0 con 0x7f9354100c50 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f935a000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93541027e0 0x7f935419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9359600700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 0x7f9354196b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f934800e970 tx=0x7f934800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348009790 con 0x7f9354100c50 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935419ec70 con 0x7f9354100c50 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f934800eec0 con 0x7f9354100c50 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348003cf0 con 0x7f9354100c50 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.129+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935419f1d0 con 0x7f9354100c50 2024-03-22T19:10:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.130+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9354108000 con 0x7f9354100c50 2024-03-22T19:10:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.132+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f934800ccd0 con 0x7f9354100c50 2024-03-22T19:10:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.133+0000 7f9352000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f933c0a0b20 0x7f933c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.134+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9348014070 con 0x7f9354100c50 2024-03-22T19:10:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.134+0000 7f9358c00700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f933c0a0b20 0x7f933c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.134+0000 7f9358c00700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f933c0a0b20 0x7f933c0a2fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9340006010 tx=0x7f934000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:35.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.137+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9348095cb0 con 0x7f9354100c50 2024-03-22T19:10:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.397+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f935402ccb0 con 0x7f9354100c50 2024-03-22T19:10:35.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.398+0000 7f9352000700 1 -- 172.21.15.86:0/3493222599 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f93480992d0 con 0x7f9354100c50 2024-03-22T19:10:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:35.400 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:09:21.111792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:35.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.400+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f933c0a0b20 msgr2=0x7f933c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.400+0000 7f935c000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f933c0a0b20 0x7f933c0a2fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9340006010 tx=0x7f934000b540 comp rx=0 tx=0).stop 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.400+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 msgr2=0x7f9354196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.400+0000 7f935c000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 0x7f9354196b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f934800e970 tx=0x7f934800ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 shutdown_connections 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f933c0a0b20 0x7f933c0a2fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93541027e0 0x7f935419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354100c50 0x7f9354196b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 --2- 172.21.15.86:0/3493222599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9354101e40 0x7f93541970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 >> 172.21.15.86:0/3493222599 conn(0x7f93540fc1c0 msgr2=0x7f9354105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 shutdown_connections 2024-03-22T19:10:35.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.401+0000 7f935c000700 1 -- 172.21.15.86:0/3493222599 wait complete. 2024-03-22T19:10:35.404 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T19:10:35.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:35.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:35 smithi086 ceph-mon[38032]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:35.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:35 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3662865125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:35.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:35 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3653261000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:35 smithi167 ceph-mon[47728]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:35 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3662865125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:35 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3653261000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 -- 172.21.15.86:0/680899629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c94101e40 msgr2=0x7f7c941022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/680899629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c94101e40 0x7f7c941022a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c800099c0 tx=0x7f7c80009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 -- 172.21.15.86:0/680899629 shutdown_connections 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/680899629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c941027e0 0x7f7c94109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/680899629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c94100c50 0x7f7c94101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/680899629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c94101e40 0x7f7c941022a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 -- 172.21.15.86:0/680899629 >> 172.21.15.86:0/680899629 conn(0x7f7c940fc1c0 msgr2=0x7f7c940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 -- 172.21.15.86:0/680899629 shutdown_connections 2024-03-22T19:10:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.611+0000 7f7c9ae00700 1 -- 172.21.15.86:0/680899629 wait complete. 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.612+0000 7f7c9ae00700 1 Processor -- start 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.612+0000 7f7c9ae00700 1 -- start start 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c9ae00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c94100c50 0x7f7c94198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c9ae00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c94101e40 0x7f7c94199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c9ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 0x7f7c941a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c9ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c941a0930 con 0x7f7c941027e0 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c9ae00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c941a0a70 con 0x7f7c94101e40 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c9ae00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c941a0c00 con 0x7f7c94100c50 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 0x7f7c941a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 0x7f7c941a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51290/0 (socket says 172.21.15.86:51290) 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 -- 172.21.15.86:0/1800793070 learned_addr learned my addr 172.21.15.86:0/1800793070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c93e00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c94100c50 0x7f7c94198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c93400700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c94101e40 0x7f7c94199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 -- 172.21.15.86:0/1800793070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c94100c50 msgr2=0x7f7c94198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c94100c50 0x7f7c94198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 -- 172.21.15.86:0/1800793070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c94101e40 msgr2=0x7f7c94199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c94101e40 0x7f7c94199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.613+0000 7f7c98e00700 1 -- 172.21.15.86:0/1800793070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800095f0 con 0x7f7c941027e0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.614+0000 7f7c98e00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 0x7f7c941a0360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00ef30 tx=0x7f7c8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.614+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c018070 con 0x7f7c941027e0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.614+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c8c00f040 con 0x7f7c941027e0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.614+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c941a0e80 con 0x7f7c941027e0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.614+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00cd00 con 0x7f7c941027e0 2024-03-22T19:10:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.614+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c941a1350 con 0x7f7c941027e0 2024-03-22T19:10:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.615+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c940679e0 con 0x7f7c941027e0 2024-03-22T19:10:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.617+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f7c8c003a00 con 0x7f7c941027e0 2024-03-22T19:10:35.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.618+0000 7f7c90c00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c780a0ba0 0x7f7c780a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:35.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.618+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7c8c0cbc00 con 0x7f7c941027e0 2024-03-22T19:10:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.618+0000 7f7c93e00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c780a0ba0 0x7f7c780a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.619+0000 7f7c93e00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c780a0ba0 0x7f7c780a3060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c94101ca0 tx=0x7f7c84007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.622+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7c8c0954f0 con 0x7f7c941027e0 2024-03-22T19:10:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:35 smithi131 ceph-mon[78559]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:10:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:35 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3662865125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:35 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3653261000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:35.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.860+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c941a16b0 con 0x7f7c941027e0 2024-03-22T19:10:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.862+0000 7f7c90c00700 1 -- 172.21.15.86:0/1800793070 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7f7c8c098b10 con 0x7f7c941027e0 2024-03-22T19:10:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:35.864 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c780a0ba0 msgr2=0x7f7c780a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c780a0ba0 0x7f7c780a3060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c94101ca0 tx=0x7f7c84007400 comp rx=0 tx=0).stop 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 msgr2=0x7f7c941a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 0x7f7c941a0360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00ef30 tx=0x7f7c8c00c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 shutdown_connections 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c780a0ba0 0x7f7c780a3060 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c941027e0 0x7f7c941a0360 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c94100c50 0x7f7c94198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 --2- 172.21.15.86:0/1800793070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c94101e40 0x7f7c94199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 >> 172.21.15.86:0/1800793070 conn(0x7f7c940fc1c0 msgr2=0x7f7c94072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 shutdown_connections 2024-03-22T19:10:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:35.865+0000 7f7c9ae00700 1 -- 172.21.15.86:0/1800793070 wait complete. 2024-03-22T19:10:35.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-22T19:09:13.618776+0000', 'modified': '2024-03-22T19:09:21.111792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24557, 'mds_2': 24550, 'mds_3': 24524, 'mds_4': 24523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14514': {'gid': 14514, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3022543840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3022543840}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3022543840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24470': {'gid': 24470, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/2878260168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 2878260168}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 2878260168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/700195904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 700195904}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 700195904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/4106390239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 4106390239}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 4106390239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/1695283976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 1695283976}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 1695283976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6835/2161889252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2161889252}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2161889252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.131:6837/3746834114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 3746834114}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 3746834114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.167:6839/3747705607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3747705607}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3747705607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6841/3747210669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3747210669}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3747210669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6839/1545428523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 1545428523}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 1545428523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-22T19:10:35.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-22T19:10:35.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 -- 172.21.15.86:0/2991153781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 msgr2=0x7f3af410b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 --2- 172.21.15.86:0/2991153781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af410b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b0a0 tx=0x7f3aec00b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 -- 172.21.15.86:0/2991153781 shutdown_connections 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 --2- 172.21.15.86:0/2991153781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af410b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 --2- 172.21.15.86:0/2991153781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3af4074df0 0x7f3af4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 --2- 172.21.15.86:0/2991153781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3af4073810 0x7f3af4073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 -- 172.21.15.86:0/2991153781 >> 172.21.15.86:0/2991153781 conn(0x7f3af40fc2c0 msgr2=0x7f3af40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 -- 172.21.15.86:0/2991153781 shutdown_connections 2024-03-22T19:10:36.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.078+0000 7f3afae00700 1 -- 172.21.15.86:0/2991153781 wait complete. 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.079+0000 7f3afae00700 1 Processor -- start 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.079+0000 7f3afae00700 1 -- start start 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3afae00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3af4073810 0x7f3af4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3afae00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3af4074df0 0x7f3af4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3afae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af41a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3afae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41a0b00 con 0x7f3af4108c10 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3afae00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41a0c40 con 0x7f3af4074df0 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3afae00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41a0dd0 con 0x7f3af4073810 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af41a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af41a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51314/0 (socket says 172.21.15.86:51314) 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 -- 172.21.15.86:0/200396785 learned_addr learned my addr 172.21.15.86:0/200396785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af3e00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3af4073810 0x7f3af4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af3400700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3af4074df0 0x7f3af4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 -- 172.21.15.86:0/200396785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3af4073810 msgr2=0x7f3af4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3af4073810 0x7f3af4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 -- 172.21.15.86:0/200396785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3af4074df0 msgr2=0x7f3af4199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3af4074df0 0x7f3af4199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.080+0000 7f3af8e00700 1 -- 172.21.15.86:0/200396785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec009d00 con 0x7f3af4108c10 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.081+0000 7f3af8e00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af41a0530 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3aec007a60 tx=0x7f3aec007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.081+0000 7f3af0c00700 1 -- 172.21.15.86:0/200396785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00e040 con 0x7f3af4108c10 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.081+0000 7f3af0c00700 1 -- 172.21.15.86:0/200396785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aec004c50 con 0x7f3af4108c10 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.081+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af41a1050 con 0x7f3af4108c10 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.081+0000 7f3af0c00700 1 -- 172.21.15.86:0/200396785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec01b6c0 con 0x7f3af4108c10 2024-03-22T19:10:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.081+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af41a1420 con 0x7f3af4108c10 2024-03-22T19:10:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.085+0000 7f3af0c00700 1 -- 172.21.15.86:0/200396785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f3aec01d470 con 0x7f3af4108c10 2024-03-22T19:10:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.086+0000 7f3b0e856b80 1 Processor -- start 2024-03-22T19:10:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.086+0000 7f3b0e856b80 1 -- start start 2024-03-22T19:10:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.086+0000 7f3af0c00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3ad80a0b20 0x7f3ad80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af0c00700 1 -- 172.21.15.86:0/200396785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3aec0cd480 con 0x7f3af4108c10 2024-03-22T19:10:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7747ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7748df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7749850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d76a0940 con 0x55b0d77493d0 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d769e300 con 0x55b0d7747bd0 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7699bf0 con 0x55b0d7748530 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af3400700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3ad80a0b20 0x7f3ad80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7749850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7749850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51320/0 (socket says 172.21.15.86:51320) 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 -- 172.21.15.86:0/683655109 learned_addr learned my addr 172.21.15.86:0/683655109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af3e00700 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7747ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 -- 172.21.15.86:0/683655109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 msgr2=0x55b0d7748df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7748df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 -- 172.21.15.86:0/683655109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 msgr2=0x55b0d7747ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7747ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af8e00700 1 -- 172.21.15.86:0/683655109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0d751b360 con 0x55b0d77493d0 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.087+0000 7f3af3e00700 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7747ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.088+0000 7f3af3400700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3ad80a0b20 0x7f3ad80a2fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3af4074b10 tx=0x7f3ae0011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.088+0000 7f3af8e00700 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7749850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0a28f0 tx=0x7f3aec00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d336cd5a1ccd2b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.088+0000 7f3ae9600700 1 -- 172.21.15.86:0/683655109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004dc0 con 0x55b0d77493d0 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.088+0000 7f3ae9600700 1 -- 172.21.15.86:0/683655109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aec09aa30 con 0x55b0d77493d0 2024-03-22T19:10:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.088+0000 7f3ae9600700 1 -- 172.21.15.86:0/683655109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec097140 con 0x55b0d77493d0 2024-03-22T19:10:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 -- 172.21.15.86:0/683655109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 msgr2=0x55b0d7749850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7749850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0a28f0 tx=0x7f3aec00b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 -- 172.21.15.86:0/683655109 shutdown_connections 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7749850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7747ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/683655109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7748df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 -- 172.21.15.86:0/683655109 >> 172.21.15.86:0/683655109 conn(0x55b0d76bd1d0 msgr2=0x55b0d7742b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 -- 172.21.15.86:0/683655109 shutdown_connections 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 -- 172.21.15.86:0/683655109 wait complete. 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 -- start start 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.089+0000 7f3b0e856b80 1 Processor -- start 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 0x55b0d7751800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7751d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d77493d0 0x55b0d7758d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7635d30 con 0x55b0d7747bd0 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d76359d0 con 0x55b0d77493d0 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d751d650 con 0x55b0d7748530 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3b0e856b80 10 client.0 _ll_register_callbacks cb 0x55b0d74a1350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 0x55b0d7751800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 0x55b0d7751800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51328/0 (socket says 172.21.15.86:51328) 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 -- 172.21.15.86:0/2645438276 learned_addr learned my addr 172.21.15.86:0/2645438276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3400700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7751d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af8e00700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d77493d0 0x55b0d7758d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 -- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 msgr2=0x55b0d7751d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7751d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 -- 172.21.15.86:0/2645438276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d77493d0 msgr2=0x55b0d7758d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d77493d0 0x55b0d7758d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3e00700 1 -- 172.21.15.86:0/2645438276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af41a1420 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.090+0000 7f3af3400700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7751d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3af3e00700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 0x55b0d7751800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4013780 tx=0x7f3ae4013b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb5263e94f05dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3ade000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3ade000700 1 -- 172.21.15.86:0/2645438276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004460 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3ade000700 1 -- 172.21.15.86:0/2645438276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3ae4003680 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0d773aa80 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3ade000700 1 -- 172.21.15.86:0/2645438276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ae4003840 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3b0e856b80 10 client.15141 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b0d74c7b10 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.091+0000 7f3ade000700 1 -- 172.21.15.86:0/2645438276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ae402a080 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3ade000700 1 -- 172.21.15.86:0/2645438276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ae4011ce0 con 0x55b0d7747bd0 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3b0e856b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3b0e856b80 10 client.15141 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3b0e856b80 20 client.15141 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d775ca10 0x55b0d7761d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3b0e856b80 4 client.15141 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0d76926b0 con 0x55b0d775ca10 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.092+0000 7f3af3400700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d775ca10 0x55b0d7761d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3ade000700 10 client.15141 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3af3400700 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d775ca10 0x55b0d7761d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89068ae2f4f8a6b7 server_cookie=d8d5a696820499e9 in_seq=0 out_seq=0 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3ade000700 1 -- 172.21.15.86:0/2645438276 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b0d76926b0 con 0x55b0d775ca10 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3ade000700 10 client.15141 handle_command_reply: tid=0 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3b0e856b80 1 client.15141 shutdown 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 msgr2=0x55b0d7751800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.093+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 0x55b0d7751800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4013780 tx=0x7f3ae4013b40 comp rx=0 tx=0).stop 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 shutdown_connections 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d77493d0 0x55b0d7758d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d7747bd0 0x55b0d7751800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d775ca10 0x55b0d7761d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/2645438276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d7751d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3ade000700 0 client.15141 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 >> 172.21.15.86:0/2645438276 conn(0x55b0d76bd1d0 msgr2=0x55b0d77421a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 shutdown_connections 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 1 -- 172.21.15.86:0/2645438276 wait complete. 2024-03-22T19:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.094+0000 7f3b0e856b80 20 client.15141 trim_cache size 0 max 16384 2024-03-22T19:10:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 Processor -- start 2024-03-22T19:10:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 -- start start 2024-03-22T19:10:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7756d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d77497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7742a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7692990 con 0x55b0d77493d0 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7696850 con 0x55b0d7747bd0 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7638de0 con 0x55b0d7748530 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7742a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.112+0000 7f3af3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7756d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7742a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51340/0 (socket says 172.21.15.86:51340) 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 -- 172.21.15.86:0/1548892491 learned_addr learned my addr 172.21.15.86:0/1548892491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af3400700 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d77497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 -- 172.21.15.86:0/1548892491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 msgr2=0x55b0d77497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d77497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 -- 172.21.15.86:0/1548892491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 msgr2=0x55b0d7756d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7756d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 -- 172.21.15.86:0/1548892491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0d765fe50 con 0x55b0d77493d0 2024-03-22T19:10:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af3e00700 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7756d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3af8e00700 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7742a70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0079c0 tx=0x7f3aec096d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2970ce2cb41d0c2b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3aea000700 1 -- 172.21.15.86:0/1548892491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004dc0 con 0x55b0d77493d0 2024-03-22T19:10:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3aea000700 1 -- 172.21.15.86:0/1548892491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aec098eb0 con 0x55b0d77493d0 2024-03-22T19:10:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.113+0000 7f3aea000700 1 -- 172.21.15.86:0/1548892491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec09ac90 con 0x55b0d77493d0 2024-03-22T19:10:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 -- 172.21.15.86:0/1548892491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 msgr2=0x55b0d7742a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7742a70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0079c0 tx=0x7f3aec096d70 comp rx=0 tx=0).stop 2024-03-22T19:10:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 -- 172.21.15.86:0/1548892491 shutdown_connections 2024-03-22T19:10:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d7742a70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7747bd0 0x55b0d7756d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/1548892491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7748530 0x55b0d77497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 -- 172.21.15.86:0/1548892491 >> 172.21.15.86:0/1548892491 conn(0x55b0d76bd1d0 msgr2=0x55b0d774f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 -- 172.21.15.86:0/1548892491 shutdown_connections 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 -- 172.21.15.86:0/1548892491 wait complete. 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.114+0000 7f3b0e856b80 1 -- start start 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 Processor -- start 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 0x55b0d7751cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7748530 0x55b0d7755d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d77562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7756930 con 0x55b0d77493d0 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d7756aa0 con 0x55b0d7748530 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d77aeff0 con 0x55b0d7747bd0 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3b0e856b80 10 client.0 _ll_register_callbacks cb 0x7f3ae4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3af3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 0x55b0d7751cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3af8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d77562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3af3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 0x55b0d7751cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:58732/0 (socket says 172.21.15.86:58732) 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3af3e00700 1 -- 172.21.15.86:0/4037744865 learned_addr learned my addr 172.21.15.86:0/4037744865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.115+0000 7f3af3400700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7748530 0x55b0d7755d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3e00700 1 -- 172.21.15.86:0/4037744865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7748530 msgr2=0x55b0d7755d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3e00700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7748530 0x55b0d7755d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3e00700 1 -- 172.21.15.86:0/4037744865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 msgr2=0x55b0d77562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3e00700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d77562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3e00700 1 -- 172.21.15.86:0/4037744865 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0d74c7b10 con 0x55b0d7747bd0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3400700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7748530 0x55b0d7755d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3af3e00700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 0x55b0d7751cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40134a0 tx=0x7f3ae4026bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4015e5edbafe477c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3ade000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3ade000700 1 -- 172.21.15.86:0/4037744865 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4005d60 con 0x55b0d7747bd0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3ade000700 1 -- 172.21.15.86:0/4037744865 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3ae4000e40 con 0x55b0d7747bd0 2024-03-22T19:10:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.116+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aec09aa30 con 0x55b0d7747bd0 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3ade000700 1 -- 172.21.15.86:0/4037744865 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ae40056e0 con 0x55b0d7747bd0 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 10 client.25177 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b0d751b360 con 0x55b0d7747bd0 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3ade000700 1 -- 172.21.15.86:0/4037744865 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aec09aa30 con 0x55b0d7747bd0 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3ade000700 1 -- 172.21.15.86:0/4037744865 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ae4003980 con 0x55b0d7747bd0 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 10 client.25177 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 10 client.25177 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 20 client.25177 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d77b3c50 0x55b0d77b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 4 client.25177 mds_command: new command op to 24523 tid=0[{"prefix": "status"}] 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.117+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b0d74e8ca0 con 0x55b0d77b3c50 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3af3400700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d77b3c50 0x55b0d77b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3ade000700 10 client.25177 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3af3400700 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d77b3c50 0x55b0d77b05c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4cc91f4f1d55677 server_cookie=a60a9702ea58e760 in_seq=0 out_seq=0 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3ade000700 1 -- 172.21.15.86:0/4037744865 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55b0d74e8ca0 con 0x55b0d77b3c50 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3ade000700 10 client.25177 handle_command_reply: tid=0 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 client.25177 shutdown 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 msgr2=0x55b0d7751cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 0x55b0d7751cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40134a0 tx=0x7f3ae4026bd0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 shutdown_connections 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0d77493d0 0x55b0d77562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55b0d77b3c50 0x55b0d77b05c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0d7747bd0 0x55b0d7751cc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3b0e856b80 1 --2- 172.21.15.86:0/4037744865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0d7748530 0x55b0d7755d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.118+0000 7f3ade000700 0 client.25177 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 >> 172.21.15.86:0/4037744865 conn(0x55b0d76bd1d0 msgr2=0x55b0d7765d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 shutdown_connections 2024-03-22T19:10:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3b0e856b80 1 -- 172.21.15.86:0/4037744865 wait complete. 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3b0e856b80 20 client.25177 trim_cache size 0 max 16384 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24523, 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 75.962286019999993, 2024-03-22T19:10:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:36.126 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:36.126 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:36.126 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 100.94443033 2024-03-22T19:10:36.126 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:36.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3ad80a0b20 msgr2=0x7f3ad80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3ad80a0b20 0x7f3ad80a2fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3af4074b10 tx=0x7f3ae0011440 comp rx=0 tx=0).stop 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 msgr2=0x7f3af41a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af41a0530 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3aec007a60 tx=0x7f3aec007930 comp rx=0 tx=0).stop 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 shutdown_connections 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3ad80a0b20 0x7f3ad80a2fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3af4108c10 0x7f3af41a0530 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3af4073810 0x7f3af4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 --2- 172.21.15.86:0/200396785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3af4074df0 0x7f3af4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.119+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 >> 172.21.15.86:0/200396785 conn(0x7f3af40fc2c0 msgr2=0x7f3af4104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.120+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 shutdown_connections 2024-03-22T19:10:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.120+0000 7f3afae00700 1 -- 172.21.15.86:0/200396785 wait complete. 2024-03-22T19:10:36.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 -- 172.21.15.86:0/1024493860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde98100d40 msgr2=0x7fde98101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 --2- 172.21.15.86:0/1024493860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde98100d40 0x7fde98101160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde880099c0 tx=0x7fde88009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 -- 172.21.15.86:0/1024493860 shutdown_connections 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 --2- 172.21.15.86:0/1024493860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde981028d0 0x7fde98109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 --2- 172.21.15.86:0/1024493860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde98100d40 0x7fde98101160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 --2- 172.21.15.86:0/1024493860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde98101f30 0x7fde98102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 -- 172.21.15.86:0/1024493860 >> 172.21.15.86:0/1024493860 conn(0x7fde980fc2b0 msgr2=0x7fde980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 -- 172.21.15.86:0/1024493860 shutdown_connections 2024-03-22T19:10:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.331+0000 7fde9ce00700 1 -- 172.21.15.86:0/1024493860 wait complete. 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 Processor -- start 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 -- start start 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde98100d40 0x7fde98074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98101f30 0x7fde980731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 0x7fde980736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98073e80 con 0x7fde981028d0 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98073ff0 con 0x7fde98101f30 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.332+0000 7fde9ce00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde981a0d80 con 0x7fde98100d40 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 0x7fde980736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 0x7fde980736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51364/0 (socket says 172.21.15.86:51364) 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 -- 172.21.15.86:0/2897522553 learned_addr learned my addr 172.21.15.86:0/2897522553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde95400700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98101f30 0x7fde980731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde95e00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde98100d40 0x7fde98074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 -- 172.21.15.86:0/2897522553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde98100d40 msgr2=0x7fde98074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde98100d40 0x7fde98074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 -- 172.21.15.86:0/2897522553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98101f30 msgr2=0x7fde980731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98101f30 0x7fde980731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 -- 172.21.15.86:0/2897522553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880095f0 con 0x7fde981028d0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.333+0000 7fde96800700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 0x7fde980736e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde90007ee0 tx=0x7fde9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.334+0000 7fde8ea00700 1 -- 172.21.15.86:0/2897522553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9000dcc0 con 0x7fde981028d0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.334+0000 7fde8ea00700 1 -- 172.21.15.86:0/2897522553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde9000f040 con 0x7fde981028d0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.334+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde981a0fb0 con 0x7fde981028d0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.334+0000 7fde8ea00700 1 -- 172.21.15.86:0/2897522553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90004830 con 0x7fde981028d0 2024-03-22T19:10:36.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.334+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde981a1490 con 0x7fde981028d0 2024-03-22T19:10:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.337+0000 7fde8ea00700 1 -- 172.21.15.86:0/2897522553 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fde90003a00 con 0x7fde981028d0 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 Processor -- start 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fde8ea00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde7c0a0b20 0x7fde7c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 -- start start 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fde8ea00700 1 -- 172.21.15.86:0/2897522553 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fde900cb9e0 con 0x7fde981028d0 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fde95400700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde7c0a0b20 0x7fde7c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b653cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b653ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b6495940 con 0x5591b653d530 2024-03-22T19:10:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b6493300 con 0x5591b653cbd0 2024-03-22T19:10:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.338+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b648ebf0 con 0x5591b653e3d0 2024-03-22T19:10:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b653cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b653ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b653cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39094/0 (socket says 172.21.15.86:39094) 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 -- 172.21.15.86:0/3856640226 learned_addr learned my addr 172.21.15.86:0/3856640226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde96800700 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95400700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde7c0a0b20 0x7fde7c0a2fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde84005950 tx=0x7fde840106b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 -- 172.21.15.86:0/3856640226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 msgr2=0x5591b653e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 -- 172.21.15.86:0/3856640226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 msgr2=0x5591b653ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b653ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95e00700 1 -- 172.21.15.86:0/3856640226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591b6310360 con 0x5591b653cbd0 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.339+0000 7fde95400700 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b653ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.340+0000 7fde95e00700 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b653cff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde880099c0 tx=0x7fde88005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd41c8727e99eba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.340+0000 7fde83400700 1 -- 172.21.15.86:0/3856640226 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8801d070 con 0x5591b653cbd0 2024-03-22T19:10:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.340+0000 7fde83400700 1 -- 172.21.15.86:0/3856640226 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8801e040 con 0x5591b653cbd0 2024-03-22T19:10:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.340+0000 7fde83400700 1 -- 172.21.15.86:0/3856640226 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde880038c0 con 0x5591b653cbd0 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.340+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3856640226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 msgr2=0x5591b653cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.340+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b653cff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde880099c0 tx=0x7fde88005bc0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3856640226 shutdown_connections 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b653cff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3856640226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b653ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3856640226 >> 172.21.15.86:0/3856640226 conn(0x5591b64b21d0 msgr2=0x5591b6537b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3856640226 shutdown_connections 2024-03-22T19:10:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3856640226 wait complete. 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 -- start start 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.341+0000 7fdeb0889b80 1 Processor -- start 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b6546aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b6546fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b654df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b642ad30 con 0x5591b653d530 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b642a9d0 con 0x5591b653cbd0 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b6312650 con 0x5591b653e3d0 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fdeb0889b80 10 client.0 _ll_register_callbacks cb 0x5591b6296350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b6546fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b6546aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b6546aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39104/0 (socket says 172.21.15.86:39104) 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 -- 172.21.15.86:0/98451640 learned_addr learned my addr 172.21.15.86:0/98451640 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde96800700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b654df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 -- 172.21.15.86:0/98451640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 msgr2=0x5591b654df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b654df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 -- 172.21.15.86:0/98451640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 msgr2=0x5591b6546fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b6546fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.342+0000 7fde95e00700 1 -- 172.21.15.86:0/98451640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde84010cc0 con 0x5591b653cbd0 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fde95400700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b6546fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fde95e00700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b6546aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde88009350 tx=0x7fde88009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11d8c22013fed135 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fde77e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fde77e00700 1 -- 172.21.15.86:0/98451640 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8801d070 con 0x5591b653cbd0 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fde77e00700 1 -- 172.21.15.86:0/98451640 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fde880055d0 con 0x5591b653cbd0 2024-03-22T19:10:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591b652fa80 con 0x5591b653cbd0 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fde77e00700 1 -- 172.21.15.86:0/98451640 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde880038c0 con 0x5591b653cbd0 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fdeb0889b80 10 client.25184 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.343+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fde98109ad0 con 0x5591b653cbd0 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fde77e00700 1 -- 172.21.15.86:0/98451640 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8803d020 con 0x5591b653cbd0 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fde77e00700 1 -- 172.21.15.86:0/98451640 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fde88004860 con 0x5591b653cbd0 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fdeb0889b80 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fdeb0889b80 10 client.25184 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fdeb0889b80 20 client.25184 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b6551690 0x5591b6556990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fdeb0889b80 4 client.25184 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591b64876b0 con 0x5591b6551690 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.344+0000 7fde95400700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b6551690 0x5591b6556990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.345+0000 7fde77e00700 10 client.25184 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.345+0000 7fde95400700 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b6551690 0x5591b6556990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a51a5e97b80994d6 server_cookie=1c36ded03be1785d in_seq=0 out_seq=0 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.345+0000 7fde77e00700 1 -- 172.21.15.86:0/98451640 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5591b64876b0 con 0x5591b6551690 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.345+0000 7fde77e00700 10 client.25184 handle_command_reply: tid=0 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 client.25184 shutdown 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 msgr2=0x5591b6546aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b6546aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde88009350 tx=0x7fde88009580 comp rx=0 tx=0).stop 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 shutdown_connections 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b6551690 0x5591b6556990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b654df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b6546aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/98451640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b6546fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fde77e00700 0 client.25184 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:36.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 >> 172.21.15.86:0/98451640 conn(0x5591b64b21d0 msgr2=0x5591b65371a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 shutdown_connections 2024-03-22T19:10:36.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 1 -- 172.21.15.86:0/98451640 wait complete. 2024-03-22T19:10:36.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.346+0000 7fdeb0889b80 20 client.25184 trim_cache size 0 max 16384 2024-03-22T19:10:36.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 Processor -- start 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 -- start start 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b654bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b654e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b644cad0 con 0x5591b653d530 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b6487990 con 0x5591b653cbd0 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b648b850 con 0x5591b653e3d0 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fde95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b654e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fde95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b654e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51388/0 (socket says 172.21.15.86:51388) 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fde95400700 1 -- 172.21.15.86:0/874507096 learned_addr learned my addr 172.21.15.86:0/874507096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fde95e00700 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b654bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.364+0000 7fde96800700 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde95400700 1 -- 172.21.15.86:0/874507096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 msgr2=0x5591b653e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde95400700 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde95400700 1 -- 172.21.15.86:0/874507096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 msgr2=0x5591b654bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde95400700 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b654bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde95400700 1 -- 172.21.15.86:0/874507096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591b62bcb10 con 0x5591b653d530 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde95400700 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b654e630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde84010fe0 tx=0x7fde84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef5f0972d38356d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde83e00700 1 -- 172.21.15.86:0/874507096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84024040 con 0x5591b653d530 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde83e00700 1 -- 172.21.15.86:0/874507096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde84029780 con 0x5591b653d530 2024-03-22T19:10:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.365+0000 7fde83e00700 1 -- 172.21.15.86:0/874507096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84005240 con 0x5591b653d530 2024-03-22T19:10:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 -- 172.21.15.86:0/874507096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 msgr2=0x5591b654e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b654e630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde84010fe0 tx=0x7fde84015000 comp rx=0 tx=0).stop 2024-03-22T19:10:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 -- 172.21.15.86:0/874507096 shutdown_connections 2024-03-22T19:10:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653e3d0 0x5591b653e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653cbd0 0x5591b654bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/874507096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653d530 0x5591b654e630 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 -- 172.21.15.86:0/874507096 >> 172.21.15.86:0/874507096 conn(0x5591b64b21d0 msgr2=0x5591b65339e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 -- 172.21.15.86:0/874507096 shutdown_connections 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 -- 172.21.15.86:0/874507096 wait complete. 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 -- start start 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.366+0000 7fdeb0889b80 1 Processor -- start 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653cbd0 0x5591b654b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653d530 0x5591b654bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 0x5591b654a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b654a760 con 0x5591b653e3d0 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b654a8d0 con 0x5591b653d530 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b654aa90 con 0x5591b653cbd0 2024-03-22T19:10:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fdeb0889b80 10 client.0 _ll_register_callbacks cb 0x7fde90007f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 0x5591b654a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 0x5591b654a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51404/0 (socket says 172.21.15.86:51404) 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 -- 172.21.15.86:0/3796980637 learned_addr learned my addr 172.21.15.86:0/3796980637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde95400700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653d530 0x5591b654bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde95e00700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653cbd0 0x5591b654b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 -- 172.21.15.86:0/3796980637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653cbd0 msgr2=0x5591b654b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653cbd0 0x5591b654b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 -- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653d530 msgr2=0x5591b654bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653d530 0x5591b654bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde96800700 1 -- 172.21.15.86:0/3796980637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde84010e50 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.367+0000 7fde95400700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653d530 0x5591b654bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde96800700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 0x5591b654a140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde90098e90 tx=0x7fde90098950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c1d64722c532b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde77e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde77e00700 1 -- 172.21.15.86:0/3796980637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90099360 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde77e00700 1 -- 172.21.15.86:0/3796980637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fde9009cd30 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde8801e040 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde77e00700 1 -- 172.21.15.86:0/3796980637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde900a1380 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 10 client.15165 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5591b6454e50 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde77e00700 1 -- 172.21.15.86:0/3796980637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde90063070 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fde77e00700 1 -- 172.21.15.86:0/3796980637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fde90004bd0 con 0x5591b653e3d0 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 10 client.15165 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 20 client.15165 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b65a84f0 0x5591b65a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 4 client.15165 mds_command: new command op to 24524 tid=0[{"prefix": "status"}] 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.368+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5591b62ddca0 con 0x5591b65a84f0 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.369+0000 7fde95e00700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b65a84f0 0x5591b65a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.369+0000 7fde95e00700 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b65a84f0 0x5591b65a4e40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21a0ff299b1ca44a server_cookie=44efeecfb49083ca in_seq=0 out_seq=0 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.369+0000 7fde77e00700 10 client.15165 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.369+0000 7fde77e00700 1 -- 172.21.15.86:0/3796980637 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5591b62ddca0 con 0x5591b65a84f0 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.369+0000 7fde77e00700 10 client.15165 handle_command_reply: tid=0 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.369+0000 7fdeb0889b80 1 client.15165 shutdown 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 msgr2=0x5591b654a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 0x5591b654a140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde90098e90 tx=0x7fde90098950 comp rx=0 tx=0).stop 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 shutdown_connections 2024-03-22T19:10:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591b653e3d0 0x5591b654a140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5591b65a84f0 0x5591b65a4e40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591b653cbd0 0x5591b654b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 --2- 172.21.15.86:0/3796980637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591b653d530 0x5591b654bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fde77e00700 0 client.15165 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 >> 172.21.15.86:0/3796980637 conn(0x5591b64b21d0 msgr2=0x5591b6545f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 shutdown_connections 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 1 -- 172.21.15.86:0/3796980637 wait complete. 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fdeb0889b80 20 client.15165 trim_cache size 0 max 16384 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:36.377 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24524, 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 77.226008585000002, 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 97.688047667000006 2024-03-22T19:10:36.378 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde7c0a0b20 msgr2=0x7fde7c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fde9ce00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde7c0a0b20 0x7fde7c0a2fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde84005950 tx=0x7fde840106b0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 msgr2=0x7fde980736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.370+0000 7fde9ce00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 0x7fde980736e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde90007ee0 tx=0x7fde9000d3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 shutdown_connections 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde7c0a0b20 0x7fde7c0a2fe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde981028d0 0x7fde980736e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde98100d40 0x7fde98074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 --2- 172.21.15.86:0/2897522553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98101f30 0x7fde980731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 >> 172.21.15.86:0/2897522553 conn(0x7fde980fc2b0 msgr2=0x7fde98105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 shutdown_connections 2024-03-22T19:10:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.371+0000 7fde9ce00700 1 -- 172.21.15.86:0/2897522553 wait complete. 2024-03-22T19:10:36.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:36 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3493222599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:36.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:36 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1800793070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:36.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-22T19:10:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:36 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3493222599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:36.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:36 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1800793070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.581+0000 7fc1a5400700 1 -- 172.21.15.86:0/262574491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a01028e0 msgr2=0x7fc1a0109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.581+0000 7fc1a5400700 1 --2- 172.21.15.86:0/262574491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a01028e0 0x7fc1a0109190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc19800b0d0 tx=0x7fc19800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 -- 172.21.15.86:0/262574491 shutdown_connections 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 --2- 172.21.15.86:0/262574491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a01028e0 0x7fc1a0109190 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 --2- 172.21.15.86:0/262574491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0100d50 0x7fc1a0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 --2- 172.21.15.86:0/262574491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a0101f40 0x7fc1a01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 -- 172.21.15.86:0/262574491 >> 172.21.15.86:0/262574491 conn(0x7fc1a00fc2c0 msgr2=0x7fc1a00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 -- 172.21.15.86:0/262574491 shutdown_connections 2024-03-22T19:10:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.582+0000 7fc1a5400700 1 -- 172.21.15.86:0/262574491 wait complete. 2024-03-22T19:10:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 Processor -- start 2024-03-22T19:10:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 -- start start 2024-03-22T19:10:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 0x7fc1a0199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0101f40 0x7fc1a0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a01028e0 0x7fc1a01a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01a0b00 con 0x7fc1a0100d50 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01a0c40 con 0x7fc1a0101f40 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.583+0000 7fc1a5400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01a0dd0 con 0x7fc1a01028e0 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 0x7fc1a0199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 0x7fc1a0199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51430/0 (socket says 172.21.15.86:51430) 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 -- 172.21.15.86:0/2778365698 learned_addr learned my addr 172.21.15.86:0/2778365698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19de00700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0101f40 0x7fc1a0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19f200700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a01028e0 0x7fc1a01a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 -- 172.21.15.86:0/2778365698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a01028e0 msgr2=0x7fc1a01a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a01028e0 0x7fc1a01a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 -- 172.21.15.86:0/2778365698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0101f40 msgr2=0x7fc1a0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0101f40 0x7fc1a0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 -- 172.21.15.86:0/2778365698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009d00 con 0x7fc1a0100d50 2024-03-22T19:10:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19de00700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0101f40 0x7fc1a0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19f200700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a01028e0 0x7fc1a01a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc19e800700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 0x7fc1a0199020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc19000d980 tx=0x7fc19000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc197400700 1 -- 172.21.15.86:0/2778365698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000bb60 con 0x7fc1a0100d50 2024-03-22T19:10:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a01a10b0 con 0x7fc1a0100d50 2024-03-22T19:10:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.584+0000 7fc197400700 1 -- 172.21.15.86:0/2778365698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc190007620 con 0x7fc1a0100d50 2024-03-22T19:10:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.585+0000 7fc197400700 1 -- 172.21.15.86:0/2778365698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190004a30 con 0x7fc1a0100d50 2024-03-22T19:10:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.585+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a01a15c0 con 0x7fc1a0100d50 2024-03-22T19:10:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.587+0000 7fc197400700 1 -- 172.21.15.86:0/2778365698 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc190004b90 con 0x7fc1a0100d50 2024-03-22T19:10:36.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc197400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1840a0b20 0x7fc1840a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc1b8e5ab80 1 Processor -- start 2024-03-22T19:10:36.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc197400700 1 -- 172.21.15.86:0/2778365698 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc1900cc4c0 con 0x7fc1a0100d50 2024-03-22T19:10:36.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc19de00700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1840a0b20 0x7fc1840a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc1b8e5ab80 1 -- start start 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc19de00700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1840a0b20 0x7fc1840a2fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005e10 tx=0x7fc18c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fdbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.589+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fdcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54f34940 con 0x55fe54fdc530 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54f32300 con 0x55fe54fdbbd0 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54f2dbf0 con 0x55fe54fdd3d0 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fdbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fdcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fdbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39144/0 (socket says 172.21.15.86:39144) 2024-03-22T19:10:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 -- 172.21.15.86:0/722362028 learned_addr learned my addr 172.21.15.86:0/722362028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19f200700 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 -- 172.21.15.86:0/722362028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 msgr2=0x55fe54fdd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 -- 172.21.15.86:0/722362028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 msgr2=0x55fe54fdcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fdcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 -- 172.21.15.86:0/722362028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe54daf360 con 0x55fe54fdbbd0 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19f200700 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.590+0000 7fc19e800700 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fdbff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc190096450 tx=0x7fc190010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ad1c43b1821063 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc18b400700 1 -- 172.21.15.86:0/722362028 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190004e40 con 0x55fe54fdbbd0 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc18b400700 1 -- 172.21.15.86:0/722362028 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19000bdc0 con 0x55fe54fdbbd0 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc18b400700 1 -- 172.21.15.86:0/722362028 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900cc0f0 con 0x55fe54fdbbd0 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/722362028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 msgr2=0x55fe54fdbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fdbff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc190096450 tx=0x7fc190010ee0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/722362028 shutdown_connections 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fdbff0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/722362028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fdcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.591+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/722362028 >> 172.21.15.86:0/722362028 conn(0x55fe54f511d0 msgr2=0x55fe54fd6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.592+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/722362028 shutdown_connections 2024-03-22T19:10:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.592+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/722362028 wait complete. 2024-03-22T19:10:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.592+0000 7fc1b8e5ab80 1 -- start start 2024-03-22T19:10:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.592+0000 7fc1b8e5ab80 1 Processor -- start 2024-03-22T19:10:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fe5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54ec9d30 con 0x55fe54fdc530 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54ec99d0 con 0x55fe54fdbbd0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54db1650 con 0x55fe54fdd3d0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc1b8e5ab80 10 client.0 _ll_register_callbacks cb 0x55fe54d35350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fe5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fe5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51446/0 (socket says 172.21.15.86:51446) 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 -- 172.21.15.86:0/7467998 learned_addr learned my addr 172.21.15.86:0/7467998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19e800700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19f200700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 -- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 msgr2=0x55fe54fecf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fecf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 -- 172.21.15.86:0/7467998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 msgr2=0x55fe54fe59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.593+0000 7fc19de00700 1 -- 172.21.15.86:0/7467998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a01a15c0 con 0x55fe54fdc530 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc19de00700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fe5f30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00f950 tx=0x7fc18c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fff2d83b40fd1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc17fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc17fe00700 1 -- 172.21.15.86:0/7467998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c017040 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc17fe00700 1 -- 172.21.15.86:0/7467998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc18c003880 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe54fcea80 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc17fe00700 1 -- 172.21.15.86:0/7467998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18c003a40 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc1b8e5ab80 10 client.15177 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fe54d5bb10 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc17fe00700 1 -- 172.21.15.86:0/7467998 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe54fcea80 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc17fe00700 1 -- 172.21.15.86:0/7467998 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc18c020410 con 0x55fe54fdc530 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.594+0000 7fc1b8e5ab80 10 client.15177 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.595+0000 7fc1b8e5ab80 10 client.15177 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.595+0000 7fc1b8e5ab80 20 client.15177 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.595+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe54ff0120 0x55fe54ff5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.595+0000 7fc1b8e5ab80 4 client.15177 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.595+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe54f266b0 con 0x55fe54ff0120 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.595+0000 7fc19e800700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe54ff0120 0x55fe54ff5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc19e800700 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe54ff0120 0x55fe54ff5f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed48332889114f6 server_cookie=942fd1454deb7223 in_seq=0 out_seq=0 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc17fe00700 10 client.15177 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc17fe00700 1 -- 172.21.15.86:0/7467998 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fe54f266b0 con 0x55fe54ff0120 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc17fe00700 10 client.15177 handle_command_reply: tid=0 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 client.15177 shutdown 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 msgr2=0x55fe54fe5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fe5f30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00f950 tx=0x7fc18c004750 comp rx=0 tx=0).stop 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 shutdown_connections 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe54ff0120 0x55fe54ff5f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdd3d0 0x55fe54fecf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc17fe00700 0 client.15177 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/7467998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdc530 0x55fe54fe5f30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.596+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 >> 172.21.15.86:0/7467998 conn(0x55fe54f511d0 msgr2=0x55fe54fd61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.597+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 shutdown_connections 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.597+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/7467998 wait complete. 2024-03-22T19:10:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.597+0000 7fc1b8e5ab80 20 client.15177 trim_cache size 0 max 16384 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.614+0000 7fc1b8e5ab80 1 Processor -- start 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.614+0000 7fc1b8e5ab80 1 -- start start 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdbbd0 0x55fe54feaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdc530 0x55fe54fed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54f26990 con 0x55fe54fdd3d0 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54f2a850 con 0x55fe54fdc530 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54fa18f0 con 0x55fe54fdbbd0 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51452/0 (socket says 172.21.15.86:51452) 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 -- 172.21.15.86:0/2699204959 learned_addr learned my addr 172.21.15.86:0/2699204959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19e800700 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdbbd0 0x55fe54feaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19de00700 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdc530 0x55fe54fed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 -- 172.21.15.86:0/2699204959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdbbd0 msgr2=0x55fe54feaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdbbd0 0x55fe54feaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 -- 172.21.15.86:0/2699204959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdc530 msgr2=0x55fe54fed620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdc530 0x55fe54fed620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19f200700 1 -- 172.21.15.86:0/2699204959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe54ef3e50 con 0x55fe54fdd3d0 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.615+0000 7fc19de00700 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdc530 0x55fe54fed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc19f200700 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc19800b0d0 tx=0x7fc198007630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6a2711e6b0f6dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc18be00700 1 -- 172.21.15.86:0/2699204959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800e040 con 0x55fe54fdd3d0 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc18be00700 1 -- 172.21.15.86:0/2699204959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc198003a00 con 0x55fe54fdd3d0 2024-03-22T19:10:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc18be00700 1 -- 172.21.15.86:0/2699204959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198005610 con 0x55fe54fdd3d0 2024-03-22T19:10:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/2699204959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 msgr2=0x55fe54fdd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc19800b0d0 tx=0x7fc198007630 comp rx=0 tx=0).stop 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/2699204959 shutdown_connections 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdd3d0 0x55fe54fdd7b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdbbd0 0x55fe54feaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/2699204959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdc530 0x55fe54fed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.616+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/2699204959 >> 172.21.15.86:0/2699204959 conn(0x55fe54f511d0 msgr2=0x55fe54fe4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/2699204959 shutdown_connections 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/2699204959 wait complete. 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 -- start start 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 Processor -- start 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdc530 0x55fe54fea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 0x55fe54fea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54feb030 con 0x55fe54fdbbd0 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe54feb1a0 con 0x55fe54fdd3d0 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.617+0000 7fc1b8e5ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55043200 con 0x55fe54fdc530 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc1b8e5ab80 10 client.0 _ll_register_callbacks cb 0x55fe54d35350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 0x55fe54fea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdc530 0x55fe54fea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 0x55fe54fea940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39174/0 (socket says 172.21.15.86:39174) 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdc530 0x55fe54fea400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:58864/0 (socket says 172.21.15.86:58864) 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 -- 172.21.15.86:0/1440497403 learned_addr learned my addr 172.21.15.86:0/1440497403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 -- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdc530 msgr2=0x55fe54fea400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdc530 0x55fe54fea400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 -- 172.21.15.86:0/1440497403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdbbd0 msgr2=0x55fe54fe9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 -- 172.21.15.86:0/1440497403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a0109ae0 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc19f200700 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 0x55fe54fea940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc198009a20 tx=0x7fc198009150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fad4f58b7b59800 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.618+0000 7fc17fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc17fe00700 1 -- 172.21.15.86:0/1440497403 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800e040 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe54fd7d20 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc17fe00700 1 -- 172.21.15.86:0/1440497403 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc1980055a0 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc17fe00700 1 -- 172.21.15.86:0/1440497403 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc198005760 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 10 client.25208 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fe54db3c80 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc17fe00700 1 -- 172.21.15.86:0/1440497403 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe54fd7d20 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc17fe00700 1 -- 172.21.15.86:0/1440497403 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc19800b100 con 0x55fe54fdd3d0 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 10 client.25208 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 10 client.25208 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 20 client.25208 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe55047bb0 0x55fe55044510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 4 client.25208 mds_command: new command op to 24550 tid=0[{"prefix": "status"}] 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.619+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fc19000bdc0 con 0x55fe55047bb0 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.620+0000 7fc19e800700 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe55047bb0 0x55fe55044510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.620+0000 7fc19e800700 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe55047bb0 0x55fe55044510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=739663b2de7f298d server_cookie=e84cf44a07443bcf in_seq=0 out_seq=0 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.620+0000 7fc17fe00700 10 client.25208 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.620+0000 7fc17fe00700 1 -- 172.21.15.86:0/1440497403 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fc19000bdc0 con 0x55fe55047bb0 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.620+0000 7fc17fe00700 10 client.25208 handle_command_reply: tid=0 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 client.25208 shutdown 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 msgr2=0x55fe54fea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 0x55fe54fea940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc198009a20 tx=0x7fc198009150 comp rx=0 tx=0).stop 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 shutdown_connections 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55fe55047bb0 0x55fe55044510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe54fdd3d0 0x55fe54fea940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe54fdbbd0 0x55fe54fe9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 --2- 172.21.15.86:0/1440497403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe54fdc530 0x55fe54fea400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc17fe00700 0 client.25208 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 >> 172.21.15.86:0/1440497403 conn(0x55fe54f511d0 msgr2=0x55fe54fe3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 shutdown_connections 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 1 -- 172.21.15.86:0/1440497403 wait complete. 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.621+0000 7fc1b8e5ab80 20 client.25208 trim_cache size 0 max 16384 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24550, 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:36.627 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 78.485569330000004, 2024-03-22T19:10:36.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:36.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:36.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:36.628 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 96.030079204000003 2024-03-22T19:10:36.628 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:36.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1840a0b20 msgr2=0x7fc1840a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1840a0b20 0x7fc1840a2fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005e10 tx=0x7fc18c005d60 comp rx=0 tx=0).stop 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 msgr2=0x7fc1a0199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 0x7fc1a0199020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc19000d980 tx=0x7fc19000dd40 comp rx=0 tx=0).stop 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 shutdown_connections 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc1840a0b20 0x7fc1840a2fe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1a01028e0 0x7fc1a01a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100d50 0x7fc1a0199020 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 --2- 172.21.15.86:0/2778365698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a0101f40 0x7fc1a0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 >> 172.21.15.86:0/2778365698 conn(0x7fc1a00fc2c0 msgr2=0x7fc1a0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 shutdown_connections 2024-03-22T19:10:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.622+0000 7fc1a5400700 1 -- 172.21.15.86:0/2778365698 wait complete. 2024-03-22T19:10:36.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T19:10:36.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:36 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3493222599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:36.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:36 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1800793070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 -- 172.21.15.86:0/504722900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8010d310 msgr2=0x7f8c8010f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 --2- 172.21.15.86:0/504722900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8010d310 0x7f8c8010f700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800b0a0 tx=0x7f8c7800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 -- 172.21.15.86:0/504722900 shutdown_connections 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 --2- 172.21.15.86:0/504722900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8010d310 0x7f8c8010f700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 --2- 172.21.15.86:0/504722900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80074df0 0x7f8c80073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 --2- 172.21.15.86:0/504722900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c80073810 0x7f8c80073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 -- 172.21.15.86:0/504722900 >> 172.21.15.86:0/504722900 conn(0x7f8c800fc2b0 msgr2=0x7f8c800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 -- 172.21.15.86:0/504722900 shutdown_connections 2024-03-22T19:10:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.832+0000 7f8c87400700 1 -- 172.21.15.86:0/504722900 wait complete. 2024-03-22T19:10:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.833+0000 7f8c87400700 1 Processor -- start 2024-03-22T19:10:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.833+0000 7f8c87400700 1 -- start start 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.833+0000 7f8c87400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80073810 0x7f8c8019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.833+0000 7f8c87400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 0x7f8c8019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.833+0000 7f8c87400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c8010d310 0x7f8c801a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c87400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a4f50 con 0x7f8c80074df0 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c87400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a5090 con 0x7f8c80073810 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c87400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a5220 con 0x7f8c8010d310 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 0x7f8c8019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 0x7f8c8019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51476/0 (socket says 172.21.15.86:51476) 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 -- 172.21.15.86:0/4118139877 learned_addr learned my addr 172.21.15.86:0/4118139877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c84a00700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80073810 0x7f8c8019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c85400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c8010d310 0x7f8c801a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 -- 172.21.15.86:0/4118139877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c8010d310 msgr2=0x7f8c801a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c8010d310 0x7f8c801a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 -- 172.21.15.86:0/4118139877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80073810 msgr2=0x7f8c8019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80073810 0x7f8c8019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 -- 172.21.15.86:0/4118139877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78009d00 con 0x7f8c80074df0 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c84a00700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80073810 0x7f8c8019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c85400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c8010d310 0x7f8c801a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.834+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 0x7f8c8019d960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00e970 tx=0x7f8c6c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.835+0000 7f8c7d600700 1 -- 172.21.15.86:0/4118139877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c009790 con 0x7f8c80074df0 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.835+0000 7f8c7d600700 1 -- 172.21.15.86:0/4118139877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c0044e0 con 0x7f8c80074df0 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.835+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c801a5500 con 0x7f8c80074df0 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.835+0000 7f8c7d600700 1 -- 172.21.15.86:0/4118139877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c003e30 con 0x7f8c80074df0 2024-03-22T19:10:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.835+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801a5a10 con 0x7f8c80074df0 2024-03-22T19:10:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.838+0000 7f8c7d600700 1 -- 172.21.15.86:0/4118139877 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f8c6c004000 con 0x7f8c80074df0 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 Processor -- start 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c7d600700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8c680a0b20 0x7f8c680a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 -- start start 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c7d600700 1 -- 172.21.15.86:0/4118139877 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8c6c014070 con 0x7f8c80074df0 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c85400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8c680a0b20 0x7f8c680a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d21529cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d21529ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d21529e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2151f5940 con 0x55d21529d530 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2151f3300 con 0x55d21529cbd0 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.839+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2151eebf0 con 0x55d21529e3d0 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d21529ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d21529cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d21529cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39192/0 (socket says 172.21.15.86:39192) 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 -- 172.21.15.86:0/3421496073 learned_addr learned my addr 172.21.15.86:0/3421496073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c85400700 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d21529e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 -- 172.21.15.86:0/3421496073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 msgr2=0x55d21529e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d21529e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 -- 172.21.15.86:0/3421496073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 msgr2=0x55d21529ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d21529ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 -- 172.21.15.86:0/3421496073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d215070360 con 0x55d21529cbd0 2024-03-22T19:10:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d21529ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c85400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8c680a0b20 0x7f8c680a2fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800b780 tx=0x7f8c780074e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.840+0000 7f8c84a00700 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d21529cff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009de0 tx=0x7f8c740116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983688af67b7b8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c71c00700 1 -- 172.21.15.86:0/3421496073 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74011e20 con 0x55d21529cbd0 2024-03-22T19:10:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c71c00700 1 -- 172.21.15.86:0/3421496073 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c74012910 con 0x55d21529cbd0 2024-03-22T19:10:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c71c00700 1 -- 172.21.15.86:0/3421496073 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400f8a0 con 0x55d21529cbd0 2024-03-22T19:10:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/3421496073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 msgr2=0x55d21529cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d21529cff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009de0 tx=0x7f8c740116c0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/3421496073 shutdown_connections 2024-03-22T19:10:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d21529e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d21529cff0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/3421496073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d21529ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/3421496073 >> 172.21.15.86:0/3421496073 conn(0x55d2152121d0 msgr2=0x55d215297b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/3421496073 shutdown_connections 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.841+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/3421496073 wait complete. 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.842+0000 7f8c9afd9b80 1 -- start start 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.842+0000 7f8c9afd9b80 1 Processor -- start 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529e3d0 0x55d2152adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21518ad30 con 0x55d21529e3d0 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21518a9d0 con 0x55d21529cbd0 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d215072650 con 0x55d21529d530 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c9afd9b80 10 client.0 _ll_register_callbacks cb 0x55d214ff6350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:58908/0 (socket says 172.21.15.86:58908) 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c7fe00700 1 -- 172.21.15.86:0/2923562687 learned_addr learned my addr 172.21.15.86:0/2923562687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c85400700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529e3d0 0x55d2152adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.843+0000 7f8c84a00700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c7fe00700 1 -- 172.21.15.86:0/2923562687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 msgr2=0x55d2152a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c7fe00700 1 -- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529e3d0 msgr2=0x55d2152adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529e3d0 0x55d2152adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c7fe00700 1 -- 172.21.15.86:0/2923562687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801a5a10 con 0x55d21529d530 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c85400700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529e3d0 0x55d2152adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152a7040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c003fd0 tx=0x7f8c6c09a170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232b74abe7442372 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c66000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c66000700 1 -- 172.21.15.86:0/2923562687 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c09d820 con 0x55d21529d530 2024-03-22T19:10:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c66000700 1 -- 172.21.15.86:0/2923562687 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8c6c0a1e70 con 0x55d21529d530 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d21528fa80 con 0x55d21529d530 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c66000700 1 -- 172.21.15.86:0/2923562687 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6c097d90 con 0x55d21529d530 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c9afd9b80 10 client.25225 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f8c74009a10 con 0x55d21529d530 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.844+0000 7f8c66000700 1 -- 172.21.15.86:0/2923562687 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d21528fa80 con 0x55d21529d530 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c66000700 1 -- 172.21.15.86:0/2923562687 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8c6c0cc070 con 0x55d21529d530 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c9afd9b80 10 client.25225 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c9afd9b80 10 client.25225 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c9afd9b80 20 client.25225 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2152b1ba0 0x55d2152b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c9afd9b80 4 client.25225 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2151e76b0 con 0x55d2152b1ba0 2024-03-22T19:10:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.845+0000 7f8c84a00700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2152b1ba0 0x55d2152b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c66000700 10 client.25225 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c84a00700 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2152b1ba0 0x55d2152b6ec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44894e7612d85d02 server_cookie=cd8f49673dfe501d in_seq=0 out_seq=0 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c66000700 1 -- 172.21.15.86:0/2923562687 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d2151e76b0 con 0x55d2152b1ba0 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c66000700 10 client.25225 handle_command_reply: tid=0 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 client.25225 shutdown 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 msgr2=0x55d2152a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152a7040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c003fd0 tx=0x7f8c6c09a170 comp rx=0 tx=0).stop 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 shutdown_connections 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2152b1ba0 0x55d2152b6ec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529e3d0 0x55d2152adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2923562687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152a7040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c66000700 0 client.25225 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.846+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 >> 172.21.15.86:0/2923562687 conn(0x55d2152121d0 msgr2=0x55d2152971a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.847+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 shutdown_connections 2024-03-22T19:10:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.847+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2923562687 wait complete. 2024-03-22T19:10:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.847+0000 7f8c9afd9b80 20 client.25225 trim_cache size 0 max 16384 2024-03-22T19:10:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.864+0000 7f8c9afd9b80 1 Processor -- start 2024-03-22T19:10:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.864+0000 7f8c9afd9b80 1 -- start start 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 0x55d2152abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529e3d0 0x55d21529e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2151acb00 con 0x55d21529cbd0 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2151e79c0 con 0x55d21529e3d0 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21518dde0 con 0x55d21529d530 2024-03-22T19:10:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 0x55d2152abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 0x55d2152abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51518/0 (socket says 172.21.15.86:51518) 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 -- 172.21.15.86:0/2401876001 learned_addr learned my addr 172.21.15.86:0/2401876001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c85400700 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529e3d0 0x55d21529e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 -- 172.21.15.86:0/2401876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 msgr2=0x55d2152ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 -- 172.21.15.86:0/2401876001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529e3d0 msgr2=0x55d21529e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529e3d0 0x55d21529e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.865+0000 7f8c84a00700 1 -- 172.21.15.86:0/2401876001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d21501cb10 con 0x55d21529cbd0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c84a00700 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 0x55d2152abf90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d215299550 tx=0x7f8c74005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dcf6c6b0283516 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c72600700 1 -- 172.21.15.86:0/2401876001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74029150 con 0x55d21529cbd0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c72600700 1 -- 172.21.15.86:0/2401876001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c740292b0 con 0x55d21529cbd0 2024-03-22T19:10:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c72600700 1 -- 172.21.15.86:0/2401876001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74029be0 con 0x55d21529cbd0 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2401876001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 msgr2=0x55d2152abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 0x55d2152abf90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d215299550 tx=0x7f8c74005b40 comp rx=0 tx=0).stop 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2401876001 shutdown_connections 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529e3d0 0x55d21529e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529cbd0 0x55d2152abf90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/2401876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529d530 0x55d2152ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2401876001 >> 172.21.15.86:0/2401876001 conn(0x55d2152121d0 msgr2=0x55d2152a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.866+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2401876001 shutdown_connections 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/2401876001 wait complete. 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 -- start start 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 Processor -- start 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d2152aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d2152ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2152a7520 con 0x55d21529d530 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2152aba50 con 0x55d21529cbd0 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2152abbe0 con 0x55d21529e3d0 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.867+0000 7f8c9afd9b80 10 client.0 _ll_register_callbacks cb 0x55d214ff6350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d2152aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d2152aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51532/0 (socket says 172.21.15.86:51532) 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 -- 172.21.15.86:0/63669548 learned_addr learned my addr 172.21.15.86:0/63669548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c84a00700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c85400700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d2152ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 -- 172.21.15.86:0/63669548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 msgr2=0x55d2152ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d2152ab510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 -- 172.21.15.86:0/63669548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 msgr2=0x55d2152a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 -- 172.21.15.86:0/63669548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d215070360 con 0x55d21529d530 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c7fe00700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d2152aafd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c095770 tx=0x7f8c6c09a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0c6f6b429ea436 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c66000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c66000700 1 -- 172.21.15.86:0/63669548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0a1e70 con 0x55d21529d530 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c66000700 1 -- 172.21.15.86:0/63669548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8c6c09d820 con 0x55d21529d530 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.868+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c74012910 con 0x55d21529d530 2024-03-22T19:10:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c66000700 1 -- 172.21.15.86:0/63669548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6c099050 con 0x55d21529d530 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 10 client.15201 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d2151b4e50 con 0x55d21529d530 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c66000700 1 -- 172.21.15.86:0/63669548 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c74012910 con 0x55d21529d530 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c66000700 1 -- 172.21.15.86:0/63669548 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8c6c01fcd0 con 0x55d21529d530 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 10 client.15201 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 20 client.15201 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2153086f0 0x55d215305060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 4 client.15201 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d21503dca0 con 0x55d2153086f0 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.869+0000 7f8c84a00700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2153086f0 0x55d215305060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c66000700 10 client.15201 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c84a00700 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2153086f0 0x55d215305060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48b68095688efce6 server_cookie=fb6f736036458f71 in_seq=0 out_seq=0 2024-03-22T19:10:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c66000700 1 -- 172.21.15.86:0/63669548 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d21503dca0 con 0x55d2153086f0 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c66000700 10 client.15201 handle_command_reply: tid=0 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 client.15201 shutdown 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 msgr2=0x55d2152aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d2152aafd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c095770 tx=0x7f8c6c09a260 comp rx=0 tx=0).stop 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 shutdown_connections 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55d2153086f0 0x55d215305060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d21529e3d0 0x55d2152ab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d21529cbd0 0x55d2152a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c66000700 0 client.15201 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 --2- 172.21.15.86:0/63669548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21529d530 0x55d2152aafd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 >> 172.21.15.86:0/63669548 conn(0x55d2152121d0 msgr2=0x55d215297a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 shutdown_connections 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.870+0000 7f8c9afd9b80 1 -- 172.21.15.86:0/63669548 wait complete. 2024-03-22T19:10:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c9afd9b80 20 client.15201 trim_cache size 0 max 16384 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24557, 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 79.740834523000004, 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:36.877 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 91.189141036999999 2024-03-22T19:10:36.878 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8c680a0b20 msgr2=0x7f8c680a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8c680a0b20 0x7f8c680a2fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800b780 tx=0x7f8c780074e0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 msgr2=0x7f8c8019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 0x7f8c8019d960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00e970 tx=0x7f8c6c00ed30 comp rx=0 tx=0).stop 2024-03-22T19:10:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 shutdown_connections 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8c680a0b20 0x7f8c680a2fe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c8010d310 0x7f8c801a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c80073810 0x7f8c8019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 --2- 172.21.15.86:0/4118139877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c80074df0 0x7f8c8019d960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 >> 172.21.15.86:0/4118139877 conn(0x7f8c800fc2b0 msgr2=0x7f8c801006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 shutdown_connections 2024-03-22T19:10:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:36.871+0000 7f8c87400700 1 -- 172.21.15.86:0/4118139877 wait complete. 2024-03-22T19:10:36.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T19:10:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 -- 172.21.15.86:0/993917711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 msgr2=0x7fa314102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 --2- 172.21.15.86:0/993917711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa314102390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa3000099c0 tx=0x7fa300009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 -- 172.21.15.86:0/993917711 shutdown_connections 2024-03-22T19:10:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 --2- 172.21.15.86:0/993917711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3141028d0 0x7fa314109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 --2- 172.21.15.86:0/993917711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa314100d40 0x7fa314101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 --2- 172.21.15.86:0/993917711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa314102390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.081+0000 7fa31aa00700 1 -- 172.21.15.86:0/993917711 >> 172.21.15.86:0/993917711 conn(0x7fa3140fc2b0 msgr2=0x7fa3140fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.082+0000 7fa31aa00700 1 -- 172.21.15.86:0/993917711 shutdown_connections 2024-03-22T19:10:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.082+0000 7fa31aa00700 1 -- 172.21.15.86:0/993917711 wait complete. 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.082+0000 7fa31aa00700 1 Processor -- start 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 -- start start 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa314100d40 0x7fa314074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa3140731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3141028d0 0x7fa3140736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa314073e80 con 0x7fa314101f30 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa314073ff0 con 0x7fa314100d40 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa31aa00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3141a0d30 con 0x7fa3141028d0 2024-03-22T19:10:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa3140731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa3140731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51552/0 (socket says 172.21.15.86:51552) 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 -- 172.21.15.86:0/2231154595 learned_addr learned my addr 172.21.15.86:0/2231154595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa318a00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3141028d0 0x7fa3140736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313e00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa314100d40 0x7fa314074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 -- 172.21.15.86:0/2231154595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3141028d0 msgr2=0x7fa3140736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3141028d0 0x7fa3140736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 -- 172.21.15.86:0/2231154595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa314100d40 msgr2=0x7fa314074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa314100d40 0x7fa314074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.083+0000 7fa313400700 1 -- 172.21.15.86:0/2231154595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000095f0 con 0x7fa314101f30 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.084+0000 7fa313400700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa3140731a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa300003730 tx=0x7fa30000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.084+0000 7fa310c00700 1 -- 172.21.15.86:0/2231154595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30001d070 con 0x7fa314101f30 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.084+0000 7fa310c00700 1 -- 172.21.15.86:0/2231154595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa300005050 con 0x7fa314101f30 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.084+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3141a0f60 con 0x7fa314101f30 2024-03-22T19:10:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.084+0000 7fa310c00700 1 -- 172.21.15.86:0/2231154595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300004b10 con 0x7fa314101f30 2024-03-22T19:10:37.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.084+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3141a1320 con 0x7fa314101f30 2024-03-22T19:10:37.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.087+0000 7fa310c00700 1 -- 172.21.15.86:0/2231154595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fa3000051c0 con 0x7fa314101f30 2024-03-22T19:10:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.088+0000 7fa310c00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa2f80a0ba0 0x7fa2f80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.088+0000 7fa32e54cb80 1 Processor -- start 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.088+0000 7fa310c00700 1 -- 172.21.15.86:0/2231154595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa3000ccf10 con 0x7fa314101f30 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 -- start start 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa313e00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa2f80a0ba0 0x7fa2f80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2022eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2022fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20230850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20187940 con 0x564a202303d0 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20185300 con 0x564a2022f530 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20180bf0 con 0x564a2022ebd0 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa318a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20230850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa313e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2022eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa318a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20230850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51568/0 (socket says 172.21.15.86:51568) 2024-03-22T19:10:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa318a00700 1 -- 172.21.15.86:0/121205115 learned_addr learned my addr 172.21.15.86:0/121205115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.089+0000 7fa313400700 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2022fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa318a00700 1 -- 172.21.15.86:0/121205115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 msgr2=0x564a2022eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa318a00700 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2022eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa318a00700 1 -- 172.21.15.86:0/121205115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 msgr2=0x564a2022fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa318a00700 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2022fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa318a00700 1 -- 172.21.15.86:0/121205115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a20002360 con 0x564a202303d0 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa313e00700 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2022eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa313e00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa2f80a0ba0 0x7fa2f80a3060 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa314101d90 tx=0x7fa304011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa318a00700 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20230850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa30c011680 tx=0x7fa30c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3198109ba675159e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa309200700 1 -- 172.21.15.86:0/121205115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00e040 con 0x564a202303d0 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa309200700 1 -- 172.21.15.86:0/121205115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30c00f040 con 0x564a202303d0 2024-03-22T19:10:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.090+0000 7fa309200700 1 -- 172.21.15.86:0/121205115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0048c0 con 0x564a202303d0 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 -- 172.21.15.86:0/121205115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 msgr2=0x564a20230850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20230850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa30c011680 tx=0x7fa30c009920 comp rx=0 tx=0).stop 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 -- 172.21.15.86:0/121205115 shutdown_connections 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20230850 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2022eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/121205115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2022fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 -- 172.21.15.86:0/121205115 >> 172.21.15.86:0/121205115 conn(0x564a201a41d0 msgr2=0x564a20229b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 -- 172.21.15.86:0/121205115 shutdown_connections 2024-03-22T19:10:37.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 -- 172.21.15.86:0/121205115 wait complete. 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.091+0000 7fa32e54cb80 1 -- start start 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 Processor -- start 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022ebd0 0x564a20238ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 0x564a20239010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a202303d0 0x564a20240000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2011cd30 con 0x564a2022f530 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2011c9d0 con 0x564a2022ebd0 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20004650 con 0x564a202303d0 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa32e54cb80 10 client.0 _ll_register_callbacks cb 0x564a1ff88350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa313400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 0x564a20239010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa313400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 0x564a20239010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51580/0 (socket says 172.21.15.86:51580) 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa313400700 1 -- 172.21.15.86:0/2562451467 learned_addr learned my addr 172.21.15.86:0/2562451467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa318a00700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a202303d0 0x564a20240000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.092+0000 7fa313e00700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022ebd0 0x564a20238ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa313400700 1 -- 172.21.15.86:0/2562451467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a202303d0 msgr2=0x564a20240000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa313400700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a202303d0 0x564a20240000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa313400700 1 -- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022ebd0 msgr2=0x564a20238ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa313400700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022ebd0 0x564a20238ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa313400700 1 -- 172.21.15.86:0/2562451467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa304011af0 con 0x564a2022f530 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa313400700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 0x564a20239010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa300096970 tx=0x7fa300005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4611a0534851a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa2fd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa2fd600700 1 -- 172.21.15.86:0/2562451467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30009e120 con 0x564a2022f530 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa2fd600700 1 -- 172.21.15.86:0/2562451467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa30009a4c0 con 0x564a2022f530 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a20221a80 con 0x564a2022f530 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa2fd600700 1 -- 172.21.15.86:0/2562451467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa30009e2f0 con 0x564a2022f530 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa32e54cb80 10 client.15213 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.093+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fa30c011330 con 0x564a2022f530 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa2fd600700 1 -- 172.21.15.86:0/2562451467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a20221a80 con 0x564a2022f530 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa2fd600700 1 -- 172.21.15.86:0/2562451467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3000a24b0 con 0x564a2022f530 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa32e54cb80 10 client.15213 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa32e54cb80 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa32e54cb80 20 client.15213 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a20243870 0x564a20248b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa32e54cb80 4 client.15213 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a201796b0 con 0x564a20243870 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.094+0000 7fa313e00700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a20243870 0x564a20248b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.095+0000 7fa2fd600700 10 client.15213 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.095+0000 7fa313e00700 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a20243870 0x564a20248b90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47c1f803002a2f8 server_cookie=591f48e0c557960e in_seq=0 out_seq=0 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa2fd600700 1 -- 172.21.15.86:0/2562451467 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564a201796b0 con 0x564a20243870 2024-03-22T19:10:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa2fd600700 10 client.15213 handle_command_reply: tid=0 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 client.15213 shutdown 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 msgr2=0x564a20239010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 0x564a20239010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa300096970 tx=0x7fa300005780 comp rx=0 tx=0).stop 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 shutdown_connections 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a20243870 0x564a20248b90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a202303d0 0x564a20240000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022ebd0 0x564a20238ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa2fd600700 0 client.15213 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/2562451467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022f530 0x564a20239010 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 >> 172.21.15.86:0/2562451467 conn(0x564a201a41d0 msgr2=0x564a202291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 shutdown_connections 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 1 -- 172.21.15.86:0/2562451467 wait complete. 2024-03-22T19:10:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.096+0000 7fa32e54cb80 20 client.15213 trim_cache size 0 max 16384 2024-03-22T19:10:37.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 Processor -- start 2024-03-22T19:10:37.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 -- start start 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 0x564a2023dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022f530 0x564a202307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a202303d0 0x564a20233320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20243000 con 0x564a2022ebd0 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a201796e0 con 0x564a202303d0 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.114+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20179990 con 0x564a2022f530 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 0x564a2023dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 0x564a2023dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51592/0 (socket says 172.21.15.86:51592) 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 -- 172.21.15.86:0/1467039399 learned_addr learned my addr 172.21.15.86:0/1467039399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313400700 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022f530 0x564a202307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 -- 172.21.15.86:0/1467039399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022f530 msgr2=0x564a202307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022f530 0x564a202307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa318a00700 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a202303d0 0x564a20233320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 -- 172.21.15.86:0/1467039399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a202303d0 msgr2=0x564a20233320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a202303d0 0x564a20233320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 -- 172.21.15.86:0/1467039399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1ffaeb10 con 0x564a2022ebd0 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa318a00700 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a202303d0 0x564a20233320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa313e00700 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 0x564a2023dfb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x564a20239660 tx=0x7fa304020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4fc747600ba020 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa309c00700 1 -- 172.21.15.86:0/1467039399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30402b820 con 0x564a2022ebd0 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa309c00700 1 -- 172.21.15.86:0/1467039399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30402be40 con 0x564a2022ebd0 2024-03-22T19:10:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.115+0000 7fa309c00700 1 -- 172.21.15.86:0/1467039399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30400fb20 con 0x564a2022ebd0 2024-03-22T19:10:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 -- 172.21.15.86:0/1467039399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 msgr2=0x564a2023dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 0x564a2023dfb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x564a20239660 tx=0x7fa304020000 comp rx=0 tx=0).stop 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 -- 172.21.15.86:0/1467039399 shutdown_connections 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a202303d0 0x564a20233320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2022ebd0 0x564a2023dfb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/1467039399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022f530 0x564a202307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 -- 172.21.15.86:0/1467039399 >> 172.21.15.86:0/1467039399 conn(0x564a201a41d0 msgr2=0x564a202259e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 -- 172.21.15.86:0/1467039399 shutdown_connections 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 -- 172.21.15.86:0/1467039399 wait complete. 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.116+0000 7fa32e54cb80 1 -- start start 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 Processor -- start 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2023cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2023d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20295e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2023dbe0 con 0x564a202303d0 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a202963b0 con 0x564a2022f530 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a20296570 con 0x564a2022ebd0 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa32e54cb80 10 client.0 _ll_register_callbacks cb 0x7fa30009a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa318a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20295e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa318a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20295e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51606/0 (socket says 172.21.15.86:51606) 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa318a00700 1 -- 172.21.15.86:0/390277973 learned_addr learned my addr 172.21.15.86:0/390277973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.117+0000 7fa313e00700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2023cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa313400700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2023d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa318a00700 1 -- 172.21.15.86:0/390277973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 msgr2=0x564a2023cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa318a00700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2023cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa318a00700 1 -- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 msgr2=0x564a2023d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa318a00700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2023d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa318a00700 1 -- 172.21.15.86:0/390277973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa304025040 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa318a00700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20295e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa30c011680 tx=0x7fa30c004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11ec084ef6a7851 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa2fd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa2fd600700 1 -- 172.21.15.86:0/390277973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00e040 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa30c00f040 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa2fd600700 1 -- 172.21.15.86:0/390277973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa30c007500 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa2fd600700 1 -- 172.21.15.86:0/390277973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa30c009ac0 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa32e54cb80 10 client.15219 fetch_fsmap learned FSMap version 26 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.118+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564a20146e50 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa2fd600700 1 -- 172.21.15.86:0/390277973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30c00f040 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa2fd600700 1 -- 172.21.15.86:0/390277973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa30c0047c0 con 0x564a202303d0 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa32e54cb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa32e54cb80 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa32e54cb80 20 client.15219 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a2029b2a0 0x564a20297c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa32e54cb80 4 client.15219 mds_command: new command op to 24569 tid=0[{"prefix": "status"}] 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a1ffcfca0 con 0x564a2029b2a0 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.119+0000 7fa313e00700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a2029b2a0 0x564a20297c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa2fd600700 10 client.15219 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa313e00700 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a2029b2a0 0x564a20297c30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ad8c3790eca229 server_cookie=ce19a622b371115e in_seq=0 out_seq=0 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa2fd600700 1 -- 172.21.15.86:0/390277973 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564a1ffcfca0 con 0x564a2029b2a0 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa2fd600700 10 client.15219 handle_command_reply: tid=0 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 client.15219 shutdown 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 msgr2=0x564a20295e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20295e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa30c011680 tx=0x7fa30c004110 comp rx=0 tx=0).stop 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 shutdown_connections 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x564a2029b2a0 0x564a20297c30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a202303d0 0x564a20295e70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a2022ebd0 0x564a2023cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 --2- 172.21.15.86:0/390277973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a2022f530 0x564a2023d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa2fd600700 0 client.15219 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 >> 172.21.15.86:0/390277973 conn(0x564a201a41d0 msgr2=0x564a20233320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 shutdown_connections 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.120+0000 7fa32e54cb80 1 -- 172.21.15.86:0/390277973 wait complete. 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa32e54cb80 20 client.15219 trim_cache size 0 max 16384 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "b22c63ac-e87e-11ee-95cd-87774f69a715", 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24569, 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T19:10:37.127 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 83.499169645999999, 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 89.715618809999995 2024-03-22T19:10:37.128 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa2f80a0ba0 msgr2=0x7fa2f80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa2f80a0ba0 0x7fa2f80a3060 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa314101d90 tx=0x7fa304011440 comp rx=0 tx=0).stop 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 msgr2=0x7fa3140731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa3140731a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa300003730 tx=0x7fa30000f690 comp rx=0 tx=0).stop 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 shutdown_connections 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa2f80a0ba0 0x7fa2f80a3060 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3141028d0 0x7fa3140736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa314100d40 0x7fa314074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 --2- 172.21.15.86:0/2231154595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa314101f30 0x7fa3140731a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 >> 172.21.15.86:0/2231154595 conn(0x7fa3140fc2b0 msgr2=0x7fa314105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 shutdown_connections 2024-03-22T19:10:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:37.121+0000 7fa31aa00700 1 -- 172.21.15.86:0/2231154595 wait complete. 2024-03-22T19:10:37.141 INFO:teuthology.run_tasks:Running task kclient... 2024-03-22T19:10:37.164 INFO:tasks.kclient:Mounting kernel clients... 2024-03-22T19:10:37.164 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-22T19:10:37.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:37.165 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T19:10:37.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:37.208 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-22T19:10:37.317 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-03-22T19:10:37.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T19:10:37.320 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-22T19:10:37.320 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T19:10:37.320 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:37.321 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T19:10:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:37 smithi086 ceph-mon[38032]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:37 smithi167 ceph-mon[47728]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:10:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:37 smithi131 ceph-mon[78559]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:10:38.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.872+0000 7f1b70a00700 1 -- 172.21.15.86:0/4197354501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c068780 msgr2=0x7f1b6c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.872+0000 7f1b70a00700 1 --2- 172.21.15.86:0/4197354501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c068780 0x7f1b6c068c00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009990 tx=0x7f1b58009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 -- 172.21.15.86:0/4197354501 shutdown_connections 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 --2- 172.21.15.86:0/4197354501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c069140 0x7f1b6c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 --2- 172.21.15.86:0/4197354501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b6c106ec0 0x7f1b6c1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 --2- 172.21.15.86:0/4197354501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c068780 0x7f1b6c068c00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 -- 172.21.15.86:0/4197354501 >> 172.21.15.86:0/4197354501 conn(0x7f1b6c076db0 msgr2=0x7f1b6c0771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 -- 172.21.15.86:0/4197354501 shutdown_connections 2024-03-22T19:10:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.873+0000 7f1b70a00700 1 -- 172.21.15.86:0/4197354501 wait complete. 2024-03-22T19:10:38.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 Processor -- start 2024-03-22T19:10:38.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 -- start start 2024-03-22T19:10:38.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c068780 0x7f1b6c0ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:38.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 0x7f1b6c100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b6c106ec0 0x7f1b6c100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c1a5550 con 0x7f1b6c069140 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c1a56c0 con 0x7f1b6c106ec0 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.874+0000 7f1b70a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c1a5880 con 0x7f1b6c068780 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 0x7f1b6c100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 0x7f1b6c100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40562/0 (socket says 172.21.15.86:40562) 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c068780 0x7f1b6c0ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 -- 172.21.15.86:0/3877583802 learned_addr learned my addr 172.21.15.86:0/3877583802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b6a800700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b6c106ec0 0x7f1b6c100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 -- 172.21.15.86:0/3877583802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c068780 msgr2=0x7f1b6c0ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c068780 0x7f1b6c0ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 -- 172.21.15.86:0/3877583802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b6c106ec0 msgr2=0x7f1b6c100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:38.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b6c106ec0 0x7f1b6c100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 -- 172.21.15.86:0/3877583802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580095f0 con 0x7f1b6c069140 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69e00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c068780 0x7f1b6c0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.875+0000 7f1b69400700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 0x7f1b6c100220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b58000c00 tx=0x7f1b58005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.876+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5801d070 con 0x7f1b6c069140 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.876+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b58022470 con 0x7f1b6c069140 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.876+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c1a5b00 con 0x7f1b6c069140 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.876+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58017650 con 0x7f1b6c069140 2024-03-22T19:10:38.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.876+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c1a5f10 con 0x7f1b6c069140 2024-03-22T19:10:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.877+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c109250 con 0x7f1b6c069140 2024-03-22T19:10:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.878+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f1b580225e0 con 0x7f1b6c069140 2024-03-22T19:10:38.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.881+0000 7f1b62a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b500a0b20 0x7f1b500a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:38.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.881+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1b580cd430 con 0x7f1b6c069140 2024-03-22T19:10:38.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.881+0000 7f1b69e00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b500a0b20 0x7f1b500a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:38.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.881+0000 7f1b69e00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b500a0b20 0x7f1b500a2fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009fd0 tx=0x7f1b5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:38.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:38.885+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1b58096e40 con 0x7f1b6c069140 2024-03-22T19:10:39.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.130+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b6c1011c0 con 0x7f1b6c069140 2024-03-22T19:10:39.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.130+0000 7f1b62a00700 1 -- 172.21.15.86:0/3877583802 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b58027020 con 0x7f1b6c069140 2024-03-22T19:10:39.132 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.132+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b500a0b20 msgr2=0x7f1b500a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.132+0000 7f1b70a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b500a0b20 0x7f1b500a2fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009fd0 tx=0x7f1b5c009260 comp rx=0 tx=0).stop 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.132+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 msgr2=0x7f1b6c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.132+0000 7f1b70a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 0x7f1b6c100220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b58000c00 tx=0x7f1b58005740 comp rx=0 tx=0).stop 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 shutdown_connections 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b500a0b20 0x7f1b500a2fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:39.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b6c106ec0 0x7f1b6c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:39.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b6c068780 0x7f1b6c0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:39.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 --2- 172.21.15.86:0/3877583802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c069140 0x7f1b6c100220 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:39.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 >> 172.21.15.86:0/3877583802 conn(0x7f1b6c076db0 msgr2=0x7f1b6c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:39.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 shutdown_connections 2024-03-22T19:10:39.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:39.133+0000 7f1b70a00700 1 -- 172.21.15.86:0/3877583802 wait complete. 2024-03-22T19:10:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:39 smithi086 ceph-mon[38032]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-22T19:10:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:39 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3877583802' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T19:10:39.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T19:10:39.554 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T19:10:39.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:39.554 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T19:10:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:39 smithi167 ceph-mon[47728]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-22T19:10:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:39 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3877583802' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T19:10:39.598 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T19:10:39.598 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T19:10:39.598 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T19:10:39.599 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T19:10:39.600 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41800sec preferred_lft 41800sec 2024-03-22T19:10:39.600 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T19:10:39.600 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T19:10:39.601 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T19:10:39.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:39.601 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-22T19:10:39.601 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T19:10:39.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-03-22T19:10:39.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-03-22T19:10:39.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-03-22T19:10:39.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T19:10:39.602 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T19:10:39.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:39 smithi131 ceph-mon[78559]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-22T19:10:39.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:39 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3877583802' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T19:10:40.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:40.054 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T19:10:40.121 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T19:10:40.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:40.125 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T19:10:40.182 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T19:10:40.183 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T19:10:40.183 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T19:10:40.183 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T19:10:40.183 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T19:10:40.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:40.183 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-22T19:10:40.183 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T19:10:40.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T19:10:40.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T19:10:40.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T19:10:40.184 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T19:10:40.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:40.839 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T19:10:40.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:40.883 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-03-22T19:10:40.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:40.965 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-22T19:10:40.965 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T19:10:40.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T19:10:40.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-22T19:10:40.966 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T19:10:41.145 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-22T19:10:41.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:41.145 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-22T19:10:41.145 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T19:10:41.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-22T19:10:41.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T19:10:41.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-22T19:10:41.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-22T19:10:41.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-22T19:10:41.146 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T19:10:41.359 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:41 smithi086 ceph-mon[38032]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-22T19:10:41.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:41.404 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-22T19:10:41.404 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T19:10:41.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T19:10:41.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T19:10:41.404 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T19:10:41.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T19:10:41.570 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:41.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:41 smithi167 ceph-mon[47728]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-22T19:10:41.611 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T19:10:41.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T19:10:41.612 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:41.693 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-22T19:10:41.693 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-22T19:10:41.693 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/4e72917a-ce51-40c9-b401-83d72303e636 2024-03-22T19:10:41.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:41.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/4e72917a-ce51-40c9-b401-83d72303e636 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-22T19:10:41.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:41 smithi131 ceph-mon[78559]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-22T19:10:41.833 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-22T19:10:41.834 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-03-22T19:10:42.468 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-22T19:10:42.468 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-03-22T19:10:42.468 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.131:6789,172.21.15.167:6789" 2024-03-22T19:10:42.468 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 0@b22c63ac-e87e-11ee-95cd-87774f69a715.cephfs=/volumes/qa/sv_1/4e72917a-ce51-40c9-b401-83d72303e636 2024-03-22T19:10:42.468 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.86:6789/172.21.15.131:6789/172.21.15.167:6789" will pass to kernel 2024-03-22T19:10:42.473 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T19:10:42.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:42.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:42.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:42.549 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T19:10:42.673 INFO:teuthology.orchestra.run.smithi086.stdout:client25250 2024-03-22T19:10:42.674 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:10:42.674 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T19:10:42.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:42.717 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T19:10:42.872 INFO:teuthology.orchestra.run.smithi086.stdout:client25250 2024-03-22T19:10:42.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:42.873 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T19:10:42.981 INFO:teuthology.orchestra.run.smithi086.stdout:client25250 2024-03-22T19:10:42.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:10:42.981 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715.client25250/status') 2024-03-22T19:10:43.104 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.25250 (3)172.21.15.86:0/627046498 2024-03-22T19:10:43.105 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-03-22T19:10:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-03-22T19:10:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-03-22T19:10:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:70 bytes copied, 5.1852e-05 s, 1.3 MB/s 2024-03-22T19:10:43.108 INFO:teuthology.run_tasks:Running task exec... 2024-03-22T19:10:43.118 INFO:teuthology.task.exec:Executing custom commands... 2024-03-22T19:10:43.118 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T19:10:43.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-22T19:10:43.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 -- 172.21.15.86:0/1131771685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 msgr2=0x7ffa64102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1131771685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64102340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa500099c0 tx=0x7ffa50009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 -- 172.21.15.86:0/1131771685 shutdown_connections 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1131771685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa64102880 0x7ffa64109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1131771685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa64100cf0 0x7ffa64101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1131771685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64102340 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 -- 172.21.15.86:0/1131771685 >> 172.21.15.86:0/1131771685 conn(0x7ffa640fc280 msgr2=0x7ffa640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 -- 172.21.15.86:0/1131771685 shutdown_connections 2024-03-22T19:10:43.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.308+0000 7ffa69a00700 1 -- 172.21.15.86:0/1131771685 wait complete. 2024-03-22T19:10:43.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.309+0000 7ffa69a00700 1 Processor -- start 2024-03-22T19:10:43.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.309+0000 7ffa69a00700 1 -- start start 2024-03-22T19:10:43.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa69a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa64100cf0 0x7ffa64198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:43.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa69a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa69a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa64102880 0x7ffa641a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa69a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa641a0970 con 0x7ffa64101ee0 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa69a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa641a0ab0 con 0x7ffa64100cf0 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa69a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa641a0c40 con 0x7ffa64102880 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40570/0 (socket says 172.21.15.86:40570) 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 -- 172.21.15.86:0/1159693737 learned_addr learned my addr 172.21.15.86:0/1159693737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa63400700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa64102880 0x7ffa641a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa64100cf0 0x7ffa64198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 -- 172.21.15.86:0/1159693737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa64102880 msgr2=0x7ffa641a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa64102880 0x7ffa641a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 -- 172.21.15.86:0/1159693737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa64100cf0 msgr2=0x7ffa64198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:43.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa64100cf0 0x7ffa64198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.310+0000 7ffa62000700 1 -- 172.21.15.86:0/1159693737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500095f0 con 0x7ffa64101ee0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.311+0000 7ffa62000700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64199380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa50003730 tx=0x7ffa5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.311+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5001d070 con 0x7ffa64101ee0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.311+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa50005050 con 0x7ffa64101ee0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.311+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50004b10 con 0x7ffa64101ee0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.311+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa641a0ec0 con 0x7ffa64101ee0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.311+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa641a1230 con 0x7ffa64101ee0 2024-03-22T19:10:43.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.312+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa64067c00 con 0x7ffa64101ee0 2024-03-22T19:10:43.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.314+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7ffa500051c0 con 0x7ffa64101ee0 2024-03-22T19:10:43.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.315+0000 7ffa5b400700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffa480a0ba0 0x7ffa480a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:43.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.315+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ffa500cce80 con 0x7ffa64101ee0 2024-03-22T19:10:43.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.315+0000 7ffa62a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffa480a0ba0 0x7ffa480a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:43.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.316+0000 7ffa62a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffa480a0ba0 0x7ffa480a3060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffa64101d40 tx=0x7ffa54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:43.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.319+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa50096770 con 0x7ffa64101ee0 2024-03-22T19:10:43.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:43 smithi086 ceph-mon[38032]: pgmap v278: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-22T19:10:43.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:43.572+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7ffa641a1590 con 0x7ffa64101ee0 2024-03-22T19:10:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:43 smithi167 ceph-mon[47728]: pgmap v278: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-22T19:10:43.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:43 smithi131 ceph-mon[78559]: pgmap v278: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-22T19:10:44.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.261+0000 7ffa5b400700 1 -- 172.21.15.86:0/1159693737 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7ffa50099d90 con 0x7ffa64101ee0 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffa480a0ba0 msgr2=0x7ffa480a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffa480a0ba0 0x7ffa480a3060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffa64101d40 tx=0x7ffa54009260 comp rx=0 tx=0).stop 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 msgr2=0x7ffa64199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64199380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa50003730 tx=0x7ffa5000f690 comp rx=0 tx=0).stop 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 shutdown_connections 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffa480a0ba0 0x7ffa480a3060 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa64102880 0x7ffa641a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa64100cf0 0x7ffa64198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 --2- 172.21.15.86:0/1159693737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa64101ee0 0x7ffa64199380 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.264+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 >> 172.21.15.86:0/1159693737 conn(0x7ffa640fc280 msgr2=0x7ffa64107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.265+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 shutdown_connections 2024-03-22T19:10:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.265+0000 7ffa69a00700 1 -- 172.21.15.86:0/1159693737 wait complete. 2024-03-22T19:10:44.286 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-22T19:10:44.297 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-22T19:10:44.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:44.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:44 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1159693737' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-22T19:10:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 -- 172.21.15.86:0/565168429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c1027e0 msgr2=0x7ff61c109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 --2- 172.21.15.86:0/565168429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c1027e0 0x7ff61c109090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff61400b0d0 tx=0x7ff61400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 -- 172.21.15.86:0/565168429 shutdown_connections 2024-03-22T19:10:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 --2- 172.21.15.86:0/565168429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c1027e0 0x7ff61c109090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 --2- 172.21.15.86:0/565168429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c100c50 0x7ff61c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 --2- 172.21.15.86:0/565168429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff61c101e40 0x7ff61c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 -- 172.21.15.86:0/565168429 >> 172.21.15.86:0/565168429 conn(0x7ff61c0fc1c0 msgr2=0x7ff61c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:44.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 -- 172.21.15.86:0/565168429 shutdown_connections 2024-03-22T19:10:44.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.508+0000 7ff623800700 1 -- 172.21.15.86:0/565168429 wait complete. 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 Processor -- start 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 -- start start 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 0x7ff61c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff61c101e40 0x7ff61c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c1027e0 0x7ff61c19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e810 con 0x7ff61c1027e0 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e980 con 0x7ff61c100c50 2024-03-22T19:10:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff623800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19eb40 con 0x7ff61c101e40 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff620e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 0x7ff61c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff620e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 0x7ff61c196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:49820/0 (socket says 172.21.15.86:49820) 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff621800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c1027e0 0x7ff61c19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff620e00700 1 -- 172.21.15.86:0/238325915 learned_addr learned my addr 172.21.15.86:0/238325915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.509+0000 7ff61be00700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff61c101e40 0x7ff61c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff620e00700 1 -- 172.21.15.86:0/238325915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff61c101e40 msgr2=0x7ff61c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff620e00700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff61c101e40 0x7ff61c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff620e00700 1 -- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c1027e0 msgr2=0x7ff61c19e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff620e00700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c1027e0 0x7ff61c19e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff620e00700 1 -- 172.21.15.86:0/238325915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610009520 con 0x7ff61c100c50 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff621800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c1027e0 0x7ff61c19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff620e00700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 0x7ff61c196cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff61000ea50 tx=0x7ff61000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000cbd0 con 0x7ff61c100c50 2024-03-22T19:10:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff614009d00 con 0x7ff61c100c50 2024-03-22T19:10:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61000cd30 con 0x7ff61c100c50 2024-03-22T19:10:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.510+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610018680 con 0x7ff61c100c50 2024-03-22T19:10:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.511+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c19f1a0 con 0x7ff61c100c50 2024-03-22T19:10:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.511+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c108080 con 0x7ff61c100c50 2024-03-22T19:10:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.513+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7ff610010430 con 0x7ff61c100c50 2024-03-22T19:10:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.514+0000 7ff619600700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff6040a0b20 0x7ff6040a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.515+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff610014070 con 0x7ff61c100c50 2024-03-22T19:10:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.515+0000 7ff61be00700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff6040a0b20 0x7ff6040a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.515+0000 7ff61be00700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff6040a0b20 0x7ff6040a2fe0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff61c1981a0 tx=0x7ff608009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.517+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff610096820 con 0x7ff61c100c50 2024-03-22T19:10:44.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:44 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1159693737' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-22T19:10:44.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:44 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1159693737' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-22T19:10:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.758+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff61c02d120 con 0x7ff61c100c50 2024-03-22T19:10:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.760+0000 7ff619600700 1 -- 172.21.15.86:0/238325915 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7ff610099e40 con 0x7ff61c100c50 2024-03-22T19:10:44.763 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:44.763 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff6040a0b20 msgr2=0x7ff6040a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff6040a0b20 0x7ff6040a2fe0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff61c1981a0 tx=0x7ff608009260 comp rx=0 tx=0).stop 2024-03-22T19:10:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 msgr2=0x7ff61c196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 0x7ff61c196cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff61000ea50 tx=0x7ff61000ed60 comp rx=0 tx=0).stop 2024-03-22T19:10:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 shutdown_connections 2024-03-22T19:10:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff6040a0b20 0x7ff6040a2fe0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c1027e0 0x7ff61c19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff61c100c50 0x7ff61c196cf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 --2- 172.21.15.86:0/238325915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff61c101e40 0x7ff61c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:44.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 >> 172.21.15.86:0/238325915 conn(0x7ff61c0fc1c0 msgr2=0x7ff61c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:44.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 shutdown_connections 2024-03-22T19:10:44.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:44.763+0000 7ff623800700 1 -- 172.21.15.86:0/238325915 wait complete. 2024-03-22T19:10:44.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:44.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 -- 172.21.15.86:0/48490930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 msgr2=0x7fc0d4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 --2- 172.21.15.86:0/48490930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d4102390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009920 tx=0x7fc0c0009c30 comp rx=0 tx=0).stop 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 -- 172.21.15.86:0/48490930 shutdown_connections 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 --2- 172.21.15.86:0/48490930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0d41028d0 0x7fc0d4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 --2- 172.21.15.86:0/48490930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4100d40 0x7fc0d4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 --2- 172.21.15.86:0/48490930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d4102390 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 -- 172.21.15.86:0/48490930 >> 172.21.15.86:0/48490930 conn(0x7fc0d40fc2b0 msgr2=0x7fc0d40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 -- 172.21.15.86:0/48490930 shutdown_connections 2024-03-22T19:10:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.006+0000 7fc0d9400700 1 -- 172.21.15.86:0/48490930 wait complete. 2024-03-22T19:10:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.007+0000 7fc0d9400700 1 Processor -- start 2024-03-22T19:10:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.007+0000 7fc0d9400700 1 -- start start 2024-03-22T19:10:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0d4100d40 0x7fc0d4198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d9400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d41028d0 0x7fc0d41a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d9400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41a0ab0 con 0x7fc0d4101f30 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d9400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41a0bf0 con 0x7fc0d4100d40 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d9400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41a0d80 con 0x7fc0d41028d0 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40612/0 (socket says 172.21.15.86:40612) 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 -- 172.21.15.86:0/542357453 learned_addr learned my addr 172.21.15.86:0/542357453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d3200700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d41028d0 0x7fc0d41a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d2800700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0d4100d40 0x7fc0d4198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 -- 172.21.15.86:0/542357453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d41028d0 msgr2=0x7fc0d41a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d41028d0 0x7fc0d41a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 -- 172.21.15.86:0/542357453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0d4100d40 msgr2=0x7fc0d4198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0d4100d40 0x7fc0d4198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.008+0000 7fc0d1e00700 1 -- 172.21.15.86:0/542357453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0009520 con 0x7fc0d4101f30 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.009+0000 7fc0d1e00700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d41994c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005e90 tx=0x7fc0c000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.009+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c001c070 con 0x7fc0d4101f30 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.009+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c000f7f0 con 0x7fc0d4101f30 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.009+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0003c00 con 0x7fc0d4101f30 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.009+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d41a1000 con 0x7fc0d4101f30 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.009+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d41a13d0 con 0x7fc0d4101f30 2024-03-22T19:10:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.010+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d4067ac0 con 0x7fc0d4101f30 2024-03-22T19:10:45.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.012+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fc0c0005040 con 0x7fc0d4101f30 2024-03-22T19:10:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.013+0000 7fc0cb400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0b80a0ba0 0x7fc0b80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.013+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc0c00ccd80 con 0x7fc0d4101f30 2024-03-22T19:10:45.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.013+0000 7fc0d2800700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0b80a0ba0 0x7fc0b80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.014+0000 7fc0d2800700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0b80a0ba0 0x7fc0b80a3060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4101d90 tx=0x7fc0c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:45.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.017+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0c0096670 con 0x7fc0d4101f30 2024-03-22T19:10:45.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.222+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0d41a15f0 con 0x7fc0d4101f30 2024-03-22T19:10:45.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.222+0000 7fc0cb400700 1 -- 172.21.15.86:0/542357453 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0c0099c90 con 0x7fc0d4101f30 2024-03-22T19:10:45.224 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0b80a0ba0 msgr2=0x7fc0b80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0b80a0ba0 0x7fc0b80a3060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4101d90 tx=0x7fc0c4009260 comp rx=0 tx=0).stop 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 msgr2=0x7fc0d41994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d41994c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005e90 tx=0x7fc0c000b790 comp rx=0 tx=0).stop 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 shutdown_connections 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc0b80a0ba0 0x7fc0b80a3060 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d41028d0 0x7fc0d41a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0d4100d40 0x7fc0d4198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 --2- 172.21.15.86:0/542357453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d4101f30 0x7fc0d41994c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 >> 172.21.15.86:0/542357453 conn(0x7fc0d40fc2b0 msgr2=0x7fc0d41077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 shutdown_connections 2024-03-22T19:10:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.225+0000 7fc0d9400700 1 -- 172.21.15.86:0/542357453 wait complete. 2024-03-22T19:10:45.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 -- 172.21.15.86:0/4018826221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8106a10 msgr2=0x7f69d8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 --2- 172.21.15.86:0/4018826221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8106a10 0x7f69d8108e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f69d000b0a0 tx=0x7f69d000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 -- 172.21.15.86:0/4018826221 shutdown_connections 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 --2- 172.21.15.86:0/4018826221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8106a10 0x7f69d8108e80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 --2- 172.21.15.86:0/4018826221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d81017b0 0x7f69d8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 --2- 172.21.15.86:0/4018826221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d81040e0 0x7f69d81064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 -- 172.21.15.86:0/4018826221 >> 172.21.15.86:0/4018826221 conn(0x7f69d80fb1a0 msgr2=0x7f69d80fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 -- 172.21.15.86:0/4018826221 shutdown_connections 2024-03-22T19:10:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.439+0000 7f69de800700 1 -- 172.21.15.86:0/4018826221 wait complete. 2024-03-22T19:10:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.440+0000 7f69de800700 1 Processor -- start 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.440+0000 7f69de800700 1 -- start start 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.440+0000 7f69de800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d81017b0 0x7f69d8198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.440+0000 7f69de800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 0x7f69d8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.440+0000 7f69de800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d8106a10 0x7f69d81a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69de800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a0af0 con 0x7f69d81040e0 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69de800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a0c30 con 0x7f69d8106a10 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69de800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a0dc0 con 0x7f69d81017b0 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 0x7f69d8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 0x7f69d8199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40624/0 (socket says 172.21.15.86:40624) 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 -- 172.21.15.86:0/3263552883 learned_addr learned my addr 172.21.15.86:0/3263552883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d7e00700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d8106a10 0x7f69d81a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d7400700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d81017b0 0x7f69d8198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 -- 172.21.15.86:0/3263552883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d81017b0 msgr2=0x7f69d8198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d81017b0 0x7f69d8198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 -- 172.21.15.86:0/3263552883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d8106a10 msgr2=0x7f69d81a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d8106a10 0x7f69d81a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 -- 172.21.15.86:0/3263552883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0009d00 con 0x7f69d81040e0 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d7e00700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d8106a10 0x7f69d81a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.441+0000 7f69d6a00700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 0x7f69d8199500 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e9a0 tx=0x7f69c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.442+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4009790 con 0x7f69d81040e0 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.442+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69c4010430 con 0x7f69d81040e0 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.442+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d81a10a0 con 0x7f69d81040e0 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.442+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4018680 con 0x7f69d81040e0 2024-03-22T19:10:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.442+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d81a15b0 con 0x7f69d81040e0 2024-03-22T19:10:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.443+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d8108590 con 0x7f69d81040e0 2024-03-22T19:10:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.445+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f69c40105a0 con 0x7f69d81040e0 2024-03-22T19:10:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.446+0000 7f69cfe00700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69bc0a0b20 0x7f69bc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.446+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f69c4014070 con 0x7f69d81040e0 2024-03-22T19:10:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.446+0000 7f69d7400700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69bc0a0b20 0x7f69bc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.447+0000 7f69d7400700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69bc0a0b20 0x7f69bc0a2fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009c60 tx=0x7f69c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.449+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f69c40529f0 con 0x7f69d81040e0 2024-03-22T19:10:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:45 smithi086 ceph-mon[38032]: pgmap v279: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-22T19:10:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:45 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1159693737' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-22T19:10:45.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:45 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:10:45.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:45 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/238325915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:45.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:45 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/542357453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:45 smithi167 ceph-mon[47728]: pgmap v279: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-22T19:10:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:45 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1159693737' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-22T19:10:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:45 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:10:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:45 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/238325915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:45 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/542357453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.658+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69d802cdc0 con 0x7f69d81040e0 2024-03-22T19:10:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.658+0000 7f69cfe00700 1 -- 172.21.15.86:0/3263552883 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69c4095bf0 con 0x7f69d81040e0 2024-03-22T19:10:45.660 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:45.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.660+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69bc0a0b20 msgr2=0x7f69bc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.660+0000 7f69de800700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69bc0a0b20 0x7f69bc0a2fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009c60 tx=0x7f69c8009260 comp rx=0 tx=0).stop 2024-03-22T19:10:45.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.660+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 msgr2=0x7f69d8199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.660+0000 7f69de800700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 0x7f69d8199500 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e9a0 tx=0x7f69c400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 shutdown_connections 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69bc0a0b20 0x7f69bc0a2fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69d8106a10 0x7f69d81a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d81017b0 0x7f69d8198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 --2- 172.21.15.86:0/3263552883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81040e0 0x7f69d8199500 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 >> 172.21.15.86:0/3263552883 conn(0x7f69d80fb1a0 msgr2=0x7f69d8100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 shutdown_connections 2024-03-22T19:10:45.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.661+0000 7f69de800700 1 -- 172.21.15.86:0/3263552883 wait complete. 2024-03-22T19:10:45.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:45 smithi131 ceph-mon[78559]: pgmap v279: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-22T19:10:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:45 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1159693737' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-22T19:10:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:45 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:10:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:45 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/238325915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:45 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/542357453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:45.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 -- 172.21.15.86:0/725900420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 msgr2=0x7f142c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 --2- 172.21.15.86:0/725900420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 0x7f142c109190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f142400b0d0 tx=0x7f142400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 -- 172.21.15.86:0/725900420 shutdown_connections 2024-03-22T19:10:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 --2- 172.21.15.86:0/725900420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 0x7f142c109190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 --2- 172.21.15.86:0/725900420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f142c100d50 0x7f142c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 --2- 172.21.15.86:0/725900420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c101f40 0x7f142c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 -- 172.21.15.86:0/725900420 >> 172.21.15.86:0/725900420 conn(0x7f142c0fc2c0 msgr2=0x7f142c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 -- 172.21.15.86:0/725900420 shutdown_connections 2024-03-22T19:10:45.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.875+0000 7f1434400700 1 -- 172.21.15.86:0/725900420 wait complete. 2024-03-22T19:10:45.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 Processor -- start 2024-03-22T19:10:45.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 -- start start 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 0x7f142c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f142c101f40 0x7f142c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 0x7f142c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19e900 con 0x7f142c100d50 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19ea70 con 0x7f142c1028e0 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1434400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19ec30 con 0x7f142c101f40 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.876+0000 7f1431a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 0x7f142c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 0x7f142c196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40642/0 (socket says 172.21.15.86:40642) 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 -- 172.21.15.86:0/1960095842 learned_addr learned my addr 172.21.15.86:0/1960095842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1432400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 0x7f142c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431000700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f142c101f40 0x7f142c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 -- 172.21.15.86:0/1960095842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f142c101f40 msgr2=0x7f142c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f142c101f40 0x7f142c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 -- 172.21.15.86:0/1960095842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 msgr2=0x7f142c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 0x7f142c19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 -- 172.21.15.86:0/1960095842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418009520 con 0x7f142c100d50 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1431a00700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 0x7f142c196e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f142000e970 tx=0x7f142000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420009790 con 0x7f142c100d50 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f142000eec0 con 0x7f142c100d50 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1424009d00 con 0x7f142c100d50 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.877+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420003cf0 con 0x7f142c100d50 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.878+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c19f1d0 con 0x7f142c100d50 2024-03-22T19:10:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.878+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c108100 con 0x7f142c100d50 2024-03-22T19:10:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.880+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f142000ccd0 con 0x7f142c100d50 2024-03-22T19:10:45.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.882+0000 7f142a000700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14140a0ba0 0x7f14140a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:45.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.883+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1420014070 con 0x7f142c100d50 2024-03-22T19:10:45.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.883+0000 7f1431000700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14140a0ba0 0x7f14140a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:45.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.883+0000 7f1431000700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14140a0ba0 0x7f14140a3060 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1418009d80 tx=0x7f1418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:45.886+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1420095ca0 con 0x7f142c100d50 2024-03-22T19:10:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.090+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f142c02cee0 con 0x7f142c100d50 2024-03-22T19:10:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.090+0000 7f142a000700 1 -- 172.21.15.86:0/1960095842 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14200992c0 con 0x7f142c100d50 2024-03-22T19:10:46.092 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14140a0ba0 msgr2=0x7f14140a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14140a0ba0 0x7f14140a3060 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1418009d80 tx=0x7f1418009260 comp rx=0 tx=0).stop 2024-03-22T19:10:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 msgr2=0x7f142c196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 0x7f142c196e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f142000e970 tx=0x7f142000ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 shutdown_connections 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f14140a0ba0 0x7f14140a3060 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142c1028e0 0x7f142c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c100d50 0x7f142c196e10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 --2- 172.21.15.86:0/1960095842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f142c101f40 0x7f142c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 >> 172.21.15.86:0/1960095842 conn(0x7f142c0fc2c0 msgr2=0x7f142c105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 shutdown_connections 2024-03-22T19:10:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.093+0000 7f1434400700 1 -- 172.21.15.86:0/1960095842 wait complete. 2024-03-22T19:10:46.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 -- 172.21.15.86:0/4269537929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478101e40 msgr2=0x7fe4781022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 --2- 172.21.15.86:0/4269537929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478101e40 0x7fe4781022a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4640099c0 tx=0x7fe464009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 -- 172.21.15.86:0/4269537929 shutdown_connections 2024-03-22T19:10:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 --2- 172.21.15.86:0/4269537929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4781027e0 0x7fe478109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 --2- 172.21.15.86:0/4269537929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe478100c50 0x7fe478101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 --2- 172.21.15.86:0/4269537929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478101e40 0x7fe4781022a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 -- 172.21.15.86:0/4269537929 >> 172.21.15.86:0/4269537929 conn(0x7fe4780fc1c0 msgr2=0x7fe4780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.306+0000 7fe47e400700 1 -- 172.21.15.86:0/4269537929 shutdown_connections 2024-03-22T19:10:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.307+0000 7fe47e400700 1 -- 172.21.15.86:0/4269537929 wait complete. 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.307+0000 7fe47e400700 1 Processor -- start 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 -- start start 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe478100c50 0x7fe478074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe478101e40 0x7fe4780731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 0x7fe4780736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe478073e80 con 0x7fe4781027e0 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe478073ff0 con 0x7fe478101e40 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe47e400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781a0c20 con 0x7fe478100c50 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe477e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 0x7fe4780736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe477e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 0x7fe4780736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40650/0 (socket says 172.21.15.86:40650) 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe477e00700 1 -- 172.21.15.86:0/143016924 learned_addr learned my addr 172.21.15.86:0/143016924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe477400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe478100c50 0x7fe478074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.308+0000 7fe476a00700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe478101e40 0x7fe4780731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe477e00700 1 -- 172.21.15.86:0/143016924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe478100c50 msgr2=0x7fe478074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe477e00700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe478100c50 0x7fe478074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe477e00700 1 -- 172.21.15.86:0/143016924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe478101e40 msgr2=0x7fe4780731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe477e00700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe478101e40 0x7fe4780731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe477e00700 1 -- 172.21.15.86:0/143016924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640095f0 con 0x7fe4781027e0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe477e00700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 0x7fe4780736e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe47000ef30 tx=0x7fe47000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470018070 con 0x7fe4781027e0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe47000f040 con 0x7fe4781027e0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4781a0e50 con 0x7fe4781027e0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000cca0 con 0x7fe4781027e0 2024-03-22T19:10:46.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.309+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4781a12d0 con 0x7fe4781027e0 2024-03-22T19:10:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.310+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4781080d0 con 0x7fe4781027e0 2024-03-22T19:10:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.312+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fe470003a00 con 0x7fe4781027e0 2024-03-22T19:10:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.313+0000 7fe46fe00700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe45c0a0b20 0x7fe45c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.314+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe4700cbd00 con 0x7fe4781027e0 2024-03-22T19:10:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.314+0000 7fe477400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe45c0a0b20 0x7fe45c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.314+0000 7fe477400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe45c0a0b20 0x7fe45c0a2fe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe478101ca0 tx=0x7fe468009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.318+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe4700955f0 con 0x7fe4781027e0 2024-03-22T19:10:46.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:46 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3263552883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:46.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:46 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1960095842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.560+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe47802ccb0 con 0x7fe4781027e0 2024-03-22T19:10:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.562+0000 7fe46fe00700 1 -- 172.21.15.86:0/143016924 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7fe470098c10 con 0x7fe4781027e0 2024-03-22T19:10:46.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:46.564 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe45c0a0b20 msgr2=0x7fe45c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe45c0a0b20 0x7fe45c0a2fe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe478101ca0 tx=0x7fe468009260 comp rx=0 tx=0).stop 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 msgr2=0x7fe4780736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 0x7fe4780736e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe47000ef30 tx=0x7fe47000c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 shutdown_connections 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe45c0a0b20 0x7fe45c0a2fe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4781027e0 0x7fe4780736e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe478100c50 0x7fe478074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 --2- 172.21.15.86:0/143016924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe478101e40 0x7fe4780731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.564+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 >> 172.21.15.86:0/143016924 conn(0x7fe4780fc1c0 msgr2=0x7fe478105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.565+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 shutdown_connections 2024-03-22T19:10:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.565+0000 7fe47e400700 1 -- 172.21.15.86:0/143016924 wait complete. 2024-03-22T19:10:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:46 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3263552883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:46 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1960095842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:46.585 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:46.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:46.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:46 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3263552883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:46.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:46 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1960095842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 -- 172.21.15.86:0/3898251567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58108c10 msgr2=0x7f8b5810b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/3898251567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58108c10 0x7f8b5810b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b0a0 tx=0x7f8b5000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 -- 172.21.15.86:0/3898251567 shutdown_connections 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/3898251567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58108c10 0x7f8b5810b0a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/3898251567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b58074df0 0x7f8b58073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/3898251567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b58073810 0x7f8b58073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 -- 172.21.15.86:0/3898251567 >> 172.21.15.86:0/3898251567 conn(0x7f8b580fc2c0 msgr2=0x7f8b580fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 -- 172.21.15.86:0/3898251567 shutdown_connections 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.780+0000 7f8b5ee00700 1 -- 172.21.15.86:0/3898251567 wait complete. 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.781+0000 7f8b5ee00700 1 Processor -- start 2024-03-22T19:10:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.781+0000 7f8b5ee00700 1 -- start start 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b58073810 0x7f8b58199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 0x7f8b58199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ee00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b58108c10 0x7f8b581a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0b40 con 0x7f8b58074df0 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0c80 con 0x7f8b58073810 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ee00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0e10 con 0x7f8b58108c10 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 0x7f8b58199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 0x7f8b58199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40660/0 (socket says 172.21.15.86:40660) 2024-03-22T19:10:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 -- 172.21.15.86:0/4227685817 learned_addr learned my addr 172.21.15.86:0/4227685817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57e00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b58073810 0x7f8b58199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b5ce00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b58108c10 0x7f8b581a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 -- 172.21.15.86:0/4227685817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b58108c10 msgr2=0x7f8b581a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b58108c10 0x7f8b581a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 -- 172.21.15.86:0/4227685817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b58073810 msgr2=0x7f8b58199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b58073810 0x7f8b58199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.782+0000 7f8b57400700 1 -- 172.21.15.86:0/4227685817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44009520 con 0x7f8b58074df0 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.783+0000 7f8b57400700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 0x7f8b58199550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400ea50 tx=0x7f8b4400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.783+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400cbd0 con 0x7f8b58074df0 2024-03-22T19:10:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.783+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4400cd30 con 0x7f8b58074df0 2024-03-22T19:10:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.783+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b50009d00 con 0x7f8b58074df0 2024-03-22T19:10:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.783+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44018680 con 0x7f8b58074df0 2024-03-22T19:10:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.783+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b581a1470 con 0x7f8b58074df0 2024-03-22T19:10:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.784+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b581079d0 con 0x7f8b58074df0 2024-03-22T19:10:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.786+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f8b44010430 con 0x7f8b58074df0 2024-03-22T19:10:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.788+0000 7f8b54c00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b3c0a0b20 0x7f8b3c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.788+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b44014070 con 0x7f8b58074df0 2024-03-22T19:10:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.788+0000 7f8b57e00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b3c0a0b20 0x7f8b3c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.789+0000 7f8b57e00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b3c0a0b20 0x7f8b3c0a2fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009c60 tx=0x7f8b48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:46.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:46.792+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8b44095f10 con 0x7f8b58074df0 2024-03-22T19:10:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.000+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b5802cdc0 con 0x7f8b58074df0 2024-03-22T19:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.001+0000 7f8b54c00700 1 -- 172.21.15.86:0/4227685817 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b44099530 con 0x7f8b58074df0 2024-03-22T19:10:47.003 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:47.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b3c0a0b20 msgr2=0x7f8b3c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b3c0a0b20 0x7f8b3c0a2fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009c60 tx=0x7f8b48009260 comp rx=0 tx=0).stop 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 msgr2=0x7f8b58199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 0x7f8b58199550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400ea50 tx=0x7f8b4400ed60 comp rx=0 tx=0).stop 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 shutdown_connections 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b3c0a0b20 0x7f8b3c0a2fe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b58108c10 0x7f8b581a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b58073810 0x7f8b58199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 --2- 172.21.15.86:0/4227685817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58074df0 0x7f8b58199550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 >> 172.21.15.86:0/4227685817 conn(0x7f8b580fc2c0 msgr2=0x7f8b58104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 shutdown_connections 2024-03-22T19:10:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.004+0000 7f8b5ee00700 1 -- 172.21.15.86:0/4227685817 wait complete. 2024-03-22T19:10:47.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 -- 172.21.15.86:0/2001256433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 msgr2=0x7f32bc102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 --2- 172.21.15.86:0/2001256433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 0x7f32bc102390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f32a80099c0 tx=0x7f32a8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 -- 172.21.15.86:0/2001256433 shutdown_connections 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 --2- 172.21.15.86:0/2001256433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1028d0 0x7f32bc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 --2- 172.21.15.86:0/2001256433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc100d40 0x7f32bc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 --2- 172.21.15.86:0/2001256433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 0x7f32bc102390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 -- 172.21.15.86:0/2001256433 >> 172.21.15.86:0/2001256433 conn(0x7f32bc0fc2b0 msgr2=0x7f32bc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 -- 172.21.15.86:0/2001256433 shutdown_connections 2024-03-22T19:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.223+0000 7f32c3200700 1 -- 172.21.15.86:0/2001256433 wait complete. 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.224+0000 7f32c3200700 1 Processor -- start 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.224+0000 7f32c3200700 1 -- start start 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc100d40 0x7f32bc196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c3200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 0x7f32bc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c3200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 0x7f32bc19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19e820 con 0x7f32bc100d40 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c3200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19e990 con 0x7f32bc101f30 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c3200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19eb50 con 0x7f32bc1028d0 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc100d40 0x7f32bc196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 0x7f32bc19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 0x7f32bc19e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:37020/0 (socket says 172.21.15.86:37020) 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32bb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 0x7f32bc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 -- 172.21.15.86:0/2878996643 learned_addr learned my addr 172.21.15.86:0/2878996643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 -- 172.21.15.86:0/2878996643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 msgr2=0x7f32bc197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 0x7f32bc197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 -- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc100d40 msgr2=0x7f32bc196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc100d40 0x7f32bc196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32c1200700 1 -- 172.21.15.86:0/2878996643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80095f0 con 0x7f32bc1028d0 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.225+0000 7f32bbe00700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc100d40 0x7f32bc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.226+0000 7f32c1200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 0x7f32bc19e250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32b400ef30 tx=0x7f32b400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.226+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4018070 con 0x7f32bc1028d0 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.226+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc19edd0 con 0x7f32bc1028d0 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.226+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b400f040 con 0x7f32bc1028d0 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.226+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400cd00 con 0x7f32bc1028d0 2024-03-22T19:10:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.226+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc19f250 con 0x7f32bc1028d0 2024-03-22T19:10:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.227+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc108210 con 0x7f32bc1028d0 2024-03-22T19:10:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.229+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f32b4003a00 con 0x7f32bc1028d0 2024-03-22T19:10:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.230+0000 7f32b8c00700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a00a0b20 0x7f32a00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.230+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f32b40cc070 con 0x7f32bc1028d0 2024-03-22T19:10:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.230+0000 7f32bbe00700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a00a0b20 0x7f32a00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.231+0000 7f32bbe00700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a00a0b20 0x7f32a00a2fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0fe710 tx=0x7f32ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.233+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f32b4095960 con 0x7f32bc1028d0 2024-03-22T19:10:47.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.440+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32bc02cdc0 con 0x7f32bc1028d0 2024-03-22T19:10:47.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.441+0000 7f32b8c00700 1 -- 172.21.15.86:0/2878996643 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f32b4098f80 con 0x7f32bc1028d0 2024-03-22T19:10:47.442 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a00a0b20 msgr2=0x7f32a00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a00a0b20 0x7f32a00a2fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0fe710 tx=0x7f32ac009260 comp rx=0 tx=0).stop 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 msgr2=0x7f32bc19e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 0x7f32bc19e250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32b400ef30 tx=0x7f32b400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 shutdown_connections 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a00a0b20 0x7f32a00a2fe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc1028d0 0x7f32bc19e250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc100d40 0x7f32bc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 --2- 172.21.15.86:0/2878996643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32bc101f30 0x7f32bc197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.443+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 >> 172.21.15.86:0/2878996643 conn(0x7f32bc0fc2b0 msgr2=0x7f32bc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.444+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 shutdown_connections 2024-03-22T19:10:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.444+0000 7f32c3200700 1 -- 172.21.15.86:0/2878996643 wait complete. 2024-03-22T19:10:47.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:47 smithi086 ceph-mon[38032]: pgmap v280: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:10:47.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:47 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/143016924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:47.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:47 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/4227685817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:47.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:47 smithi167 ceph-mon[47728]: pgmap v280: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:10:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:47 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/143016924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:47 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/4227685817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 -- 172.21.15.86:0/1255336078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 msgr2=0x7f4c9c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/1255336078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c102390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4c880099c0 tx=0x7f4c88009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 -- 172.21.15.86:0/1255336078 shutdown_connections 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/1255336078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c1028d0 0x7f4c9c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/1255336078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c100d40 0x7f4c9c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/1255336078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c102390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 -- 172.21.15.86:0/1255336078 >> 172.21.15.86:0/1255336078 conn(0x7f4c9c0fc2b0 msgr2=0x7f4c9c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 -- 172.21.15.86:0/1255336078 shutdown_connections 2024-03-22T19:10:47.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.657+0000 7f4ca3e00700 1 -- 172.21.15.86:0/1255336078 wait complete. 2024-03-22T19:10:47.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 Processor -- start 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 -- start start 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100d40 0x7f4c9c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c1028d0 0x7f4c9c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1a09a0 con 0x7f4c9c101f30 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1a0ae0 con 0x7f4c9c100d40 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.658+0000 7f4ca3e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1a0c70 con 0x7f4c9c1028d0 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40696/0 (socket says 172.21.15.86:40696) 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100d40 0x7f4c9c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 -- 172.21.15.86:0/662238469 learned_addr learned my addr 172.21.15.86:0/662238469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca1e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c1028d0 0x7f4c9c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 -- 172.21.15.86:0/662238469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c1028d0 msgr2=0x7f4c9c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c1028d0 0x7f4c9c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 -- 172.21.15.86:0/662238469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100d40 msgr2=0x7f4c9c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100d40 0x7f4c9c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 -- 172.21.15.86:0/662238469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880095f0 con 0x7f4c9c101f30 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca1400700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100d40 0x7f4c9c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.659+0000 7f4ca0a00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c1993b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c010 tx=0x7f4c8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.660+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8801d070 con 0x7f4c9c101f30 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.660+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c88005050 con 0x7f4c9c101f30 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.660+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c1a0ef0 con 0x7f4c9c101f30 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.660+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88004a70 con 0x7f4c9c101f30 2024-03-22T19:10:47.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.660+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c1a12b0 con 0x7f4c9c101f30 2024-03-22T19:10:47.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.661+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c067ac0 con 0x7f4c9c101f30 2024-03-22T19:10:47.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.662+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f4c88005450 con 0x7f4c9c101f30 2024-03-22T19:10:47.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.664+0000 7f4c9a000700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c840a0b20 0x7f4c840a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:47.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.664+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f4c880ccf40 con 0x7f4c9c101f30 2024-03-22T19:10:47.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.664+0000 7f4ca1400700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c840a0b20 0x7f4c840a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:47.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.665+0000 7f4ca1400700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c840a0b20 0x7f4c840a2fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c101d90 tx=0x7f4c90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.668+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4c88096830 con 0x7f4c9c101f30 2024-03-22T19:10:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:47 smithi131 ceph-mon[78559]: pgmap v280: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:10:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:47 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/143016924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:47 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/4227685817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:47.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.873+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c9c1a14a0 con 0x7f4c9c101f30 2024-03-22T19:10:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.874+0000 7f4c9a000700 1 -- 172.21.15.86:0/662238469 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c88099e50 con 0x7f4c9c101f30 2024-03-22T19:10:47.875 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c840a0b20 msgr2=0x7f4c840a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c840a0b20 0x7f4c840a2fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c101d90 tx=0x7f4c90009260 comp rx=0 tx=0).stop 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 msgr2=0x7f4c9c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c1993b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c010 tx=0x7f4c8800f690 comp rx=0 tx=0).stop 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 shutdown_connections 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f4c840a0b20 0x7f4c840a2fe0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c1028d0 0x7f4c9c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c9c100d40 0x7f4c9c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 --2- 172.21.15.86:0/662238469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9c101f30 0x7f4c9c1993b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 >> 172.21.15.86:0/662238469 conn(0x7f4c9c0fc2b0 msgr2=0x7f4c9c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 shutdown_connections 2024-03-22T19:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:47.876+0000 7f4ca3e00700 1 -- 172.21.15.86:0/662238469 wait complete. 2024-03-22T19:10:47.896 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-22T19:10:47.896 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:48.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 -- 172.21.15.86:0/2542677912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 msgr2=0x7f55381022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 --2- 172.21.15.86:0/2542677912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f55381022a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55240099c0 tx=0x7f5524009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 -- 172.21.15.86:0/2542677912 shutdown_connections 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 --2- 172.21.15.86:0/2542677912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55381027e0 0x7f5538109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 --2- 172.21.15.86:0/2542677912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5538100c50 0x7f5538101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 --2- 172.21.15.86:0/2542677912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f55381022a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 -- 172.21.15.86:0/2542677912 >> 172.21.15.86:0/2542677912 conn(0x7f55380fc1c0 msgr2=0x7f55380fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 -- 172.21.15.86:0/2542677912 shutdown_connections 2024-03-22T19:10:48.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.087+0000 7f553e600700 1 -- 172.21.15.86:0/2542677912 wait complete. 2024-03-22T19:10:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 Processor -- start 2024-03-22T19:10:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 -- start start 2024-03-22T19:10:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 0x7f5538198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f5538199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55381027e0 0x7f55381a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381a08e0 con 0x7f5538100c50 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381a0a50 con 0x7f5538101e40 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.088+0000 7f553e600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381a0c10 con 0x7f55381027e0 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 0x7f5538198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 0x7f5538198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40702/0 (socket says 172.21.15.86:40702) 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 -- 172.21.15.86:0/1434816244 learned_addr learned my addr 172.21.15.86:0/1434816244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5536a00700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f5538199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537e00700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55381027e0 0x7f55381a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 -- 172.21.15.86:0/1434816244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55381027e0 msgr2=0x7f55381a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55381027e0 0x7f55381a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 -- 172.21.15.86:0/1434816244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 msgr2=0x7f5538199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f5538199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 -- 172.21.15.86:0/1434816244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009c40 con 0x7f5538100c50 2024-03-22T19:10:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5536a00700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f5538199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f5537400700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 0x7f5538198df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f552800e970 tx=0x7f552800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.089+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528009790 con 0x7f5538100c50 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.090+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240095f0 con 0x7f5538100c50 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.090+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55280044e0 con 0x7f5538100c50 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.090+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528003e30 con 0x7f5538100c50 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.090+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55381a1200 con 0x7f5538100c50 2024-03-22T19:10:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.091+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380679e0 con 0x7f5538100c50 2024-03-22T19:10:48.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.092+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f5528004000 con 0x7f5538100c50 2024-03-22T19:10:48.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.094+0000 7f552fe00700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f551c0a0b20 0x7f551c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.095+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5528014070 con 0x7f5538100c50 2024-03-22T19:10:48.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.095+0000 7f5536a00700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f551c0a0b20 0x7f551c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.095+0000 7f5536a00700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f551c0a0b20 0x7f551c0a2fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5524000c00 tx=0x7f5524019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.098+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5528095d50 con 0x7f5538100c50 2024-03-22T19:10:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.359+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55381a1450 con 0x7f5538100c50 2024-03-22T19:10:48.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.359+0000 7f552fe00700 1 -- 172.21.15.86:0/1434816244 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f5528099370 con 0x7f5538100c50 2024-03-22T19:10:48.361 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:48.361 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:10:44.254849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f551c0a0b20 msgr2=0x7f551c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f551c0a0b20 0x7f551c0a2fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5524000c00 tx=0x7f5524019040 comp rx=0 tx=0).stop 2024-03-22T19:10:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 msgr2=0x7f5538198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 0x7f5538198df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f552800e970 tx=0x7f552800ed30 comp rx=0 tx=0).stop 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 shutdown_connections 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f551c0a0b20 0x7f551c0a2fe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55381027e0 0x7f55381a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5538100c50 0x7f5538198df0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 --2- 172.21.15.86:0/1434816244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538101e40 0x7f5538199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 >> 172.21.15.86:0/1434816244 conn(0x7f55380fc1c0 msgr2=0x7f5538072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 shutdown_connections 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.362+0000 7f553e600700 1 -- 172.21.15.86:0/1434816244 wait complete. 2024-03-22T19:10:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-22T19:10:48.381 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-22T19:10:48.382 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-22T19:10:48.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 -- 172.21.15.86:0/1451054556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 msgr2=0x7fa6441022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 --2- 172.21.15.86:0/1451054556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa6441022a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa6300099c0 tx=0x7fa630009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 -- 172.21.15.86:0/1451054556 shutdown_connections 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 --2- 172.21.15.86:0/1451054556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6441027e0 0x7fa644109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 --2- 172.21.15.86:0/1451054556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa644100c50 0x7fa644101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 --2- 172.21.15.86:0/1451054556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa6441022a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 -- 172.21.15.86:0/1451054556 >> 172.21.15.86:0/1451054556 conn(0x7fa6440fc1c0 msgr2=0x7fa6440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 -- 172.21.15.86:0/1451054556 shutdown_connections 2024-03-22T19:10:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.573+0000 7fa64ba00700 1 -- 172.21.15.86:0/1451054556 wait complete. 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 Processor -- start 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 -- start start 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa644100c50 0x7fa644198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa644199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6441027e0 0x7fa6441a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441a0930 con 0x7fa644101e40 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441a0a70 con 0x7fa644100c50 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.574+0000 7fa64ba00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441a0c00 con 0x7fa6441027e0 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa644199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa644199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42946/0 (socket says 172.21.15.86:42946) 2024-03-22T19:10:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 -- 172.21.15.86:0/2093519645 learned_addr learned my addr 172.21.15.86:0/2093519645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa649000700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa644100c50 0x7fa644198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa649a00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6441027e0 0x7fa6441a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 -- 172.21.15.86:0/2093519645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6441027e0 msgr2=0x7fa6441a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6441027e0 0x7fa6441a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 -- 172.21.15.86:0/2093519645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa644100c50 msgr2=0x7fa644198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa644100c50 0x7fa644198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 -- 172.21.15.86:0/2093519645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300095f0 con 0x7fa644101e40 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.575+0000 7fa643e00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa644199390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa63000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.576+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63001d070 con 0x7fa644101e40 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.576+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa630005030 con 0x7fa644101e40 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.576+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630004b10 con 0x7fa644101e40 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.576+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6441a0e80 con 0x7fa644101e40 2024-03-22T19:10:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.576+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6441a11f0 con 0x7fa644101e40 2024-03-22T19:10:48.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:48 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2878996643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:48.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:48 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/662238469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:48.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.577+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440679e0 con 0x7fa644101e40 2024-03-22T19:10:48.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.579+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7fa6300051a0 con 0x7fa644101e40 2024-03-22T19:10:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.580+0000 7fa641600700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa62c0a0ba0 0x7fa62c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.580+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa6300ccdf0 con 0x7fa644101e40 2024-03-22T19:10:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.580+0000 7fa649000700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa62c0a0ba0 0x7fa62c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:48.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.581+0000 7fa649000700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa62c0a0ba0 0x7fa62c0a3060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa644101ca0 tx=0x7fa638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.584+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa6300966e0 con 0x7fa644101e40 2024-03-22T19:10:48.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:48 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2878996643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:48.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:48 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/662238469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:48.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:48 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2878996643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:48.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:48 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/662238469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.785+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa644074700 con 0x7fa62c0a0ba0 2024-03-22T19:10:48.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.797+0000 7fa641600700 1 -- 172.21.15.86:0/2093519645 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344286 (secure 0 0 0) 0x7fa644074700 con 0x7fa62c0a0ba0 2024-03-22T19:10:48.806 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa62c0a0ba0 msgr2=0x7fa62c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa62c0a0ba0 0x7fa62c0a3060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa644101ca0 tx=0x7fa638009260 comp rx=0 tx=0).stop 2024-03-22T19:10:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 msgr2=0x7fa644199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa644199390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa63000f690 comp rx=0 tx=0).stop 2024-03-22T19:10:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 shutdown_connections 2024-03-22T19:10:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa62c0a0ba0 0x7fa62c0a3060 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6441027e0 0x7fa6441a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa644100c50 0x7fa644198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 --2- 172.21.15.86:0/2093519645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa644101e40 0x7fa644199390 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.801+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 >> 172.21.15.86:0/2093519645 conn(0x7fa6440fc1c0 msgr2=0x7fa644072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.802+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 shutdown_connections 2024-03-22T19:10:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:48.802+0000 7fa64ba00700 1 -- 172.21.15.86:0/2093519645 wait complete. 2024-03-22T19:10:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:10:48.822 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-03-22T19:10:48.172559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657183,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1016,"num_read_kb":628,"num_write":416,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335240,"kb_used_data":11000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606584,"statfs":{"total":1151940427776,"available":1151597142016,"internally_reserved":0,"allocated":11264000,"data_stored":7118970,"data_compressed":112425,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":612,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":64,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003269"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229812+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:50.229812+0000","last_peered":"2024-03-22T19:09:50.229812+0000","last_clean":"2024-03-22T19:09:50.229812+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:50.229812+0000","last_undegraded":"2024-03-22T19:09:50.229812+0000","last_fullsized":"2024-03-22T19:09:50.229812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225850+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:50.225850+0000","last_peered":"2024-03-22T19:09:50.225850+0000","last_clean":"2024-03-22T19:09:50.225850+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:50.225850+0000","last_undegraded":"2024-03-22T19:09:50.225850+0000","last_fullsized":"2024-03-22T19:09:50.225850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225832+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:50.225832+0000","last_peered":"2024-03-22T19:09:50.225832+0000","last_clean":"2024-03-22T19:09:50.225832+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:50.225832+0000","last_undegraded":"2024-03-22T19:09:50.225832+0000","last_fullsized":"2024-03-22T19:09:50.225832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223923+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:50.223923+0000","last_peered":"2024-03-22T19:09:50.223923+0000","last_clean":"2024-03-22T19:09:50.223923+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:50.223923+0000","last_undegraded":"2024-03-22T19:09:50.223923+0000","last_fullsized":"2024-03-22T19:09:50.223923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231033+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:50.231033+0000","last_peered":"2024-03-22T19:09:50.231033+0000","last_clean":"2024-03-22T19:09:50.231033+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:50.231033+0000","last_undegraded":"2024-03-22T19:09:50.231033+0000","last_fullsized":"2024-03-22T19:09:50.231033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227105+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:50.227105+0000","last_peered":"2024-03-22T19:09:50.227105+0000","last_clean":"2024-03-22T19:09:50.227105+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:50.227105+0000","last_undegraded":"2024-03-22T19:09:50.227105+0000","last_fullsized":"2024-03-22T19:09:50.227105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224692+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:50.224692+0000","last_peered":"2024-03-22T19:09:50.224692+0000","last_clean":"2024-03-22T19:09:50.224692+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:50.224692+0000","last_undegraded":"2024-03-22T19:09:50.224692+0000","last_fullsized":"2024-03-22T19:09:50.224692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230105+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:50.230105+0000","last_peered":"2024-03-22T19:09:50.230105+0000","last_clean":"2024-03-22T19:09:50.230105+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:50.230105+0000","last_undegraded":"2024-03-22T19:09:50.230105+0000","last_fullsized":"2024-03-22T19:09:50.230105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228937+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:50.228937+0000","last_peered":"2024-03-22T19:09:50.228937+0000","last_clean":"2024-03-22T19:09:50.228937+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:50.228937+0000","last_undegraded":"2024-03-22T19:09:50.228937+0000","last_fullsized":"2024-03-22T19:09:50.228937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280115+0000","last_change":"2024-03-22T19:09:51.280115+0000","last_active":"2024-03-22T19:09:51.280115+0000","last_peered":"2024-03-22T19:09:51.280115+0000","last_clean":"2024-03-22T19:09:51.280115+0000","last_became_active":"2024-03-22T19:09:51.279764+0000","last_became_peered":"2024-03-22T19:09:51.279764+0000","last_unstale":"2024-03-22T19:09:51.280115+0000","last_undegraded":"2024-03-22T19:09:51.280115+0000","last_fullsized":"2024-03-22T19:09:51.280115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:51:35.601327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224581+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:50.224581+0000","last_peered":"2024-03-22T19:09:50.224581+0000","last_clean":"2024-03-22T19:09:50.224581+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:50.224581+0000","last_undegraded":"2024-03-22T19:09:50.224581+0000","last_fullsized":"2024-03-22T19:09:50.224581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223728+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:50.223728+0000","last_peered":"2024-03-22T19:09:50.223728+0000","last_clean":"2024-03-22T19:09:50.223728+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:50.223728+0000","last_undegraded":"2024-03-22T19:09:50.223728+0000","last_fullsized":"2024-03-22T19:09:50.223728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221704+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:50.221704+0000","last_peered":"2024-03-22T19:09:50.221704+0000","last_clean":"2024-03-22T19:09:50.221704+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:50.221704+0000","last_undegraded":"2024-03-22T19:09:50.221704+0000","last_fullsized":"2024-03-22T19:09:50.221704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224532+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:50.224532+0000","last_peered":"2024-03-22T19:09:50.224532+0000","last_clean":"2024-03-22T19:09:50.224532+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:50.224532+0000","last_undegraded":"2024-03-22T19:09:50.224532+0000","last_fullsized":"2024-03-22T19:09:50.224532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225345+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:50.225345+0000","last_peered":"2024-03-22T19:09:50.225345+0000","last_clean":"2024-03-22T19:09:50.225345+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:50.225345+0000","last_undegraded":"2024-03-22T19:09:50.225345+0000","last_fullsized":"2024-03-22T19:09:50.225345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228414+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:50.228414+0000","last_peered":"2024-03-22T19:09:50.228414+0000","last_clean":"2024-03-22T19:09:50.228414+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:50.228414+0000","last_undegraded":"2024-03-22T19:09:50.228414+0000","last_fullsized":"2024-03-22T19:09:50.228414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229938+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:50.229938+0000","last_peered":"2024-03-22T19:09:50.229938+0000","last_clean":"2024-03-22T19:09:50.229938+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:50.229938+0000","last_undegraded":"2024-03-22T19:09:50.229938+0000","last_fullsized":"2024-03-22T19:09:50.229938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224855+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:50.224855+0000","last_peered":"2024-03-22T19:09:50.224855+0000","last_clean":"2024-03-22T19:09:50.224855+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:50.224855+0000","last_undegraded":"2024-03-22T19:09:50.224855+0000","last_fullsized":"2024-03-22T19:09:50.224855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226114+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:50.226114+0000","last_peered":"2024-03-22T19:09:50.226114+0000","last_clean":"2024-03-22T19:09:50.226114+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:50.226114+0000","last_undegraded":"2024-03-22T19:09:50.226114+0000","last_fullsized":"2024-03-22T19:09:50.226114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:44.232970+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:10:44.232970+0000","last_peered":"2024-03-22T19:10:44.232970+0000","last_clean":"2024-03-22T19:10:44.232970+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:10:44.232970+0000","last_undegraded":"2024-03-22T19:10:44.232970+0000","last_fullsized":"2024-03-22T19:10:44.232970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230751+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:50.230751+0000","last_peered":"2024-03-22T19:09:50.230751+0000","last_clean":"2024-03-22T19:09:50.230751+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:50.230751+0000","last_undegraded":"2024-03-22T19:09:50.230751+0000","last_fullsized":"2024-03-22T19:09:50.230751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230850+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:50.230850+0000","last_peered":"2024-03-22T19:09:50.230850+0000","last_clean":"2024-03-22T19:09:50.230850+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:50.230850+0000","last_undegraded":"2024-03-22T19:09:50.230850+0000","last_fullsized":"2024-03-22T19:09:50.230850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226153+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:50.226153+0000","last_peered":"2024-03-22T19:09:50.226153+0000","last_clean":"2024-03-22T19:09:50.226153+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:50.226153+0000","last_undegraded":"2024-03-22T19:09:50.226153+0000","last_fullsized":"2024-03-22T19:09:50.226153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231344+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:50.231344+0000","last_peered":"2024-03-22T19:09:50.231344+0000","last_clean":"2024-03-22T19:09:50.231344+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:50.231344+0000","last_undegraded":"2024-03-22T19:09:50.231344+0000","last_fullsized":"2024-03-22T19:09:50.231344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228584+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:50.228584+0000","last_peered":"2024-03-22T19:09:50.228584+0000","last_clean":"2024-03-22T19:09:50.228584+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:50.228584+0000","last_undegraded":"2024-03-22T19:09:50.228584+0000","last_fullsized":"2024-03-22T19:09:50.228584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231109+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:50.231109+0000","last_peered":"2024-03-22T19:09:50.231109+0000","last_clean":"2024-03-22T19:09:50.231109+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:50.231109+0000","last_undegraded":"2024-03-22T19:09:50.231109+0000","last_fullsized":"2024-03-22T19:09:50.231109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232324+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:50.232324+0000","last_peered":"2024-03-22T19:09:50.232324+0000","last_clean":"2024-03-22T19:09:50.232324+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:50.232324+0000","last_undegraded":"2024-03-22T19:09:50.232324+0000","last_fullsized":"2024-03-22T19:09:50.232324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:45.234575+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:10:45.234575+0000","last_peered":"2024-03-22T19:10:45.234575+0000","last_clean":"2024-03-22T19:10:45.234575+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:10:45.234575+0000","last_undegraded":"2024-03-22T19:10:45.234575+0000","last_fullsized":"2024-03-22T19:10:45.234575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232010+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:50.232010+0000","last_peered":"2024-03-22T19:09:50.232010+0000","last_clean":"2024-03-22T19:09:50.232010+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:50.232010+0000","last_undegraded":"2024-03-22T19:09:50.232010+0000","last_fullsized":"2024-03-22T19:09:50.232010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227866+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:50.227866+0000","last_peered":"2024-03-22T19:09:50.227866+0000","last_clean":"2024-03-22T19:09:50.227866+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:50.227866+0000","last_undegraded":"2024-03-22T19:09:50.227866+0000","last_fullsized":"2024-03-22T19:09:50.227866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230805+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:50.230805+0000","last_peered":"2024-03-22T19:09:50.230805+0000","last_clean":"2024-03-22T19:09:50.230805+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:50.230805+0000","last_undegraded":"2024-03-22T19:09:50.230805+0000","last_fullsized":"2024-03-22T19:09:50.230805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221505+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:50.221505+0000","last_peered":"2024-03-22T19:09:50.221505+0000","last_clean":"2024-03-22T19:09:50.221505+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:50.221505+0000","last_undegraded":"2024-03-22T19:09:50.221505+0000","last_fullsized":"2024-03-22T19:09:50.221505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230067+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:50.230067+0000","last_peered":"2024-03-22T19:09:50.230067+0000","last_clean":"2024-03-22T19:09:50.230067+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:50.230067+0000","last_undegraded":"2024-03-22T19:09:50.230067+0000","last_fullsized":"2024-03-22T19:09:50.230067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229198+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:50.229198+0000","last_peered":"2024-03-22T19:09:50.229198+0000","last_clean":"2024-03-22T19:09:50.229198+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:50.229198+0000","last_undegraded":"2024-03-22T19:09:50.229198+0000","last_fullsized":"2024-03-22T19:09:50.229198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224276+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:50.224276+0000","last_peered":"2024-03-22T19:09:50.224276+0000","last_clean":"2024-03-22T19:09:50.224276+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:50.224276+0000","last_undegraded":"2024-03-22T19:09:50.224276+0000","last_fullsized":"2024-03-22T19:09:50.224276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":190,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:46.235367+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:10:46.235367+0000","last_peered":"2024-03-22T19:10:46.235367+0000","last_clean":"2024-03-22T19:10:46.235367+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:10:46.235367+0000","last_undegraded":"2024-03-22T19:10:46.235367+0000","last_fullsized":"2024-03-22T19:10:46.235367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228903+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:50.228903+0000","last_peered":"2024-03-22T19:09:50.228903+0000","last_clean":"2024-03-22T19:09:50.228903+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:50.228903+0000","last_undegraded":"2024-03-22T19:09:50.228903+0000","last_fullsized":"2024-03-22T19:09:50.228903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.438753+0000","last_change":"2024-03-22T19:09:51.438753+0000","last_active":"2024-03-22T19:09:51.438753+0000","last_peered":"2024-03-22T19:09:51.438753+0000","last_clean":"2024-03-22T19:09:51.438753+0000","last_became_active":"2024-03-22T19:09:51.413301+0000","last_became_peered":"2024-03-22T19:09:51.413301+0000","last_unstale":"2024-03-22T19:09:51.438753+0000","last_undegraded":"2024-03-22T19:09:51.438753+0000","last_fullsized":"2024-03-22T19:09:51.438753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:49.994282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226106+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:50.226106+0000","last_peered":"2024-03-22T19:09:50.226106+0000","last_clean":"2024-03-22T19:09:50.226106+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:50.226106+0000","last_undegraded":"2024-03-22T19:09:50.226106+0000","last_fullsized":"2024-03-22T19:09:50.226106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232420+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:50.232420+0000","last_peered":"2024-03-22T19:09:50.232420+0000","last_clean":"2024-03-22T19:09:50.232420+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:50.232420+0000","last_undegraded":"2024-03-22T19:09:50.232420+0000","last_fullsized":"2024-03-22T19:09:50.232420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228700+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:50.228700+0000","last_peered":"2024-03-22T19:09:50.228700+0000","last_clean":"2024-03-22T19:09:50.228700+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:50.228700+0000","last_undegraded":"2024-03-22T19:09:50.228700+0000","last_fullsized":"2024-03-22T19:09:50.228700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":223,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:46.250848+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:10:46.250848+0000","last_peered":"2024-03-22T19:10:46.250848+0000","last_clean":"2024-03-22T19:10:46.250848+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:10:46.250848+0000","last_undegraded":"2024-03-22T19:10:46.250848+0000","last_fullsized":"2024-03-22T19:10:46.250848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59653,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":148,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230888+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:50.230888+0000","last_peered":"2024-03-22T19:09:50.230888+0000","last_clean":"2024-03-22T19:09:50.230888+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:50.230888+0000","last_undegraded":"2024-03-22T19:09:50.230888+0000","last_fullsized":"2024-03-22T19:09:50.230888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226121+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:50.226121+0000","last_peered":"2024-03-22T19:09:50.226121+0000","last_clean":"2024-03-22T19:09:50.226121+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:50.226121+0000","last_undegraded":"2024-03-22T19:09:50.226121+0000","last_fullsized":"2024-03-22T19:09:50.226121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221747+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:50.221747+0000","last_peered":"2024-03-22T19:09:50.221747+0000","last_clean":"2024-03-22T19:09:50.221747+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:50.221747+0000","last_undegraded":"2024-03-22T19:09:50.221747+0000","last_fullsized":"2024-03-22T19:09:50.221747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230590+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:50.230590+0000","last_peered":"2024-03-22T19:09:50.230590+0000","last_clean":"2024-03-22T19:09:50.230590+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:50.230590+0000","last_undegraded":"2024-03-22T19:09:50.230590+0000","last_fullsized":"2024-03-22T19:09:50.230590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224672+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:50.224672+0000","last_peered":"2024-03-22T19:09:50.224672+0000","last_clean":"2024-03-22T19:09:50.224672+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:50.224672+0000","last_undegraded":"2024-03-22T19:09:50.224672+0000","last_fullsized":"2024-03-22T19:09:50.224672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:43.232065+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:10:43.232065+0000","last_peered":"2024-03-22T19:10:43.232065+0000","last_clean":"2024-03-22T19:10:43.232065+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:10:43.232065+0000","last_undegraded":"2024-03-22T19:10:43.232065+0000","last_fullsized":"2024-03-22T19:10:43.232065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.435708+0000","last_change":"2024-03-22T19:09:51.435708+0000","last_active":"2024-03-22T19:09:51.435708+0000","last_peered":"2024-03-22T19:09:51.435708+0000","last_clean":"2024-03-22T19:09:51.435708+0000","last_became_active":"2024-03-22T19:09:51.412550+0000","last_became_peered":"2024-03-22T19:09:51.412550+0000","last_unstale":"2024-03-22T19:09:51.435708+0000","last_undegraded":"2024-03-22T19:09:51.435708+0000","last_fullsized":"2024-03-22T19:09:51.435708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:11:44.092847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229832+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:50.229832+0000","last_peered":"2024-03-22T19:09:50.229832+0000","last_clean":"2024-03-22T19:09:50.229832+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:50.229832+0000","last_undegraded":"2024-03-22T19:09:50.229832+0000","last_fullsized":"2024-03-22T19:09:50.229832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231493+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:50.231493+0000","last_peered":"2024-03-22T19:09:50.231493+0000","last_clean":"2024-03-22T19:09:50.231493+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:50.231493+0000","last_undegraded":"2024-03-22T19:09:50.231493+0000","last_fullsized":"2024-03-22T19:09:50.231493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229589+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:50.229589+0000","last_peered":"2024-03-22T19:09:50.229589+0000","last_clean":"2024-03-22T19:09:50.229589+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:50.229589+0000","last_undegraded":"2024-03-22T19:09:50.229589+0000","last_fullsized":"2024-03-22T19:09:50.229589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225716+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:50.225716+0000","last_peered":"2024-03-22T19:09:50.225716+0000","last_clean":"2024-03-22T19:09:50.225716+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:50.225716+0000","last_undegraded":"2024-03-22T19:09:50.225716+0000","last_fullsized":"2024-03-22T19:09:50.225716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227355+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:50.227355+0000","last_peered":"2024-03-22T19:09:50.227355+0000","last_clean":"2024-03-22T19:09:50.227355+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:50.227355+0000","last_undegraded":"2024-03-22T19:09:50.227355+0000","last_fullsized":"2024-03-22T19:09:50.227355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:45.234724+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:10:45.234724+0000","last_peered":"2024-03-22T19:10:45.234724+0000","last_clean":"2024-03-22T19:10:45.234724+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:10:45.234724+0000","last_undegraded":"2024-03-22T19:10:45.234724+0000","last_fullsized":"2024-03-22T19:10:45.234724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227464+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:50.227464+0000","last_peered":"2024-03-22T19:09:50.227464+0000","last_clean":"2024-03-22T19:09:50.227464+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:50.227464+0000","last_undegraded":"2024-03-22T19:09:50.227464+0000","last_fullsized":"2024-03-22T19:09:50.227464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229382+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:50.229382+0000","last_peered":"2024-03-22T19:09:50.229382+0000","last_clean":"2024-03-22T19:09:50.229382+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:50.229382+0000","last_undegraded":"2024-03-22T19:09:50.229382+0000","last_fullsized":"2024-03-22T19:09:50.229382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:43.231386+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:10:43.231386+0000","last_peered":"2024-03-22T19:10:43.231386+0000","last_clean":"2024-03-22T19:10:43.231386+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:10:43.231386+0000","last_undegraded":"2024-03-22T19:10:43.231386+0000","last_fullsized":"2024-03-22T19:10:43.231386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230745+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:50.230745+0000","last_peered":"2024-03-22T19:09:50.230745+0000","last_clean":"2024-03-22T19:09:50.230745+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:50.230745+0000","last_undegraded":"2024-03-22T19:09:50.230745+0000","last_fullsized":"2024-03-22T19:09:50.230745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223407+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:50.223407+0000","last_peered":"2024-03-22T19:09:50.223407+0000","last_clean":"2024-03-22T19:09:50.223407+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:50.223407+0000","last_undegraded":"2024-03-22T19:09:50.223407+0000","last_fullsized":"2024-03-22T19:09:50.223407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224404+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:50.224404+0000","last_peered":"2024-03-22T19:09:50.224404+0000","last_clean":"2024-03-22T19:09:50.224404+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:50.224404+0000","last_undegraded":"2024-03-22T19:09:50.224404+0000","last_fullsized":"2024-03-22T19:09:50.224404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221409+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:50.221409+0000","last_peered":"2024-03-22T19:09:50.221409+0000","last_clean":"2024-03-22T19:09:50.221409+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:50.221409+0000","last_undegraded":"2024-03-22T19:09:50.221409+0000","last_fullsized":"2024-03-22T19:09:50.221409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224573+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:50.224573+0000","last_peered":"2024-03-22T19:09:50.224573+0000","last_clean":"2024-03-22T19:09:50.224573+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:50.224573+0000","last_undegraded":"2024-03-22T19:09:50.224573+0000","last_fullsized":"2024-03-22T19:09:50.224573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231242+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:50.231242+0000","last_peered":"2024-03-22T19:09:50.231242+0000","last_clean":"2024-03-22T19:09:50.231242+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:50.231242+0000","last_undegraded":"2024-03-22T19:09:50.231242+0000","last_fullsized":"2024-03-22T19:09:50.231242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221846+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:50.221846+0000","last_peered":"2024-03-22T19:09:50.221846+0000","last_clean":"2024-03-22T19:09:50.221846+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:50.221846+0000","last_undegraded":"2024-03-22T19:09:50.221846+0000","last_fullsized":"2024-03-22T19:09:50.221846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224775+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:50.224775+0000","last_peered":"2024-03-22T19:09:50.224775+0000","last_clean":"2024-03-22T19:09:50.224775+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:50.224775+0000","last_undegraded":"2024-03-22T19:09:50.224775+0000","last_fullsized":"2024-03-22T19:09:50.224775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230722+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:50.230722+0000","last_peered":"2024-03-22T19:09:50.230722+0000","last_clean":"2024-03-22T19:09:50.230722+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:50.230722+0000","last_undegraded":"2024-03-22T19:09:50.230722+0000","last_fullsized":"2024-03-22T19:09:50.230722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223816+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:50.223816+0000","last_peered":"2024-03-22T19:09:50.223816+0000","last_clean":"2024-03-22T19:09:50.223816+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:50.223816+0000","last_undegraded":"2024-03-22T19:09:50.223816+0000","last_fullsized":"2024-03-22T19:09:50.223816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225947+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:50.225947+0000","last_peered":"2024-03-22T19:09:50.225947+0000","last_clean":"2024-03-22T19:09:50.225947+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:50.225947+0000","last_undegraded":"2024-03-22T19:09:50.225947+0000","last_fullsized":"2024-03-22T19:09:50.225947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228471+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:50.228471+0000","last_peered":"2024-03-22T19:09:50.228471+0000","last_clean":"2024-03-22T19:09:50.228471+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:50.228471+0000","last_undegraded":"2024-03-22T19:09:50.228471+0000","last_fullsized":"2024-03-22T19:09:50.228471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230778+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:50.230778+0000","last_peered":"2024-03-22T19:09:50.230778+0000","last_clean":"2024-03-22T19:09:50.230778+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:50.230778+0000","last_undegraded":"2024-03-22T19:09:50.230778+0000","last_fullsized":"2024-03-22T19:09:50.230778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230201+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:50.230201+0000","last_peered":"2024-03-22T19:09:50.230201+0000","last_clean":"2024-03-22T19:09:50.230201+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:50.230201+0000","last_undegraded":"2024-03-22T19:09:50.230201+0000","last_fullsized":"2024-03-22T19:09:50.230201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229252+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:50.229252+0000","last_peered":"2024-03-22T19:09:50.229252+0000","last_clean":"2024-03-22T19:09:50.229252+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:50.229252+0000","last_undegraded":"2024-03-22T19:09:50.229252+0000","last_fullsized":"2024-03-22T19:09:50.229252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.241586+0000","last_change":"2024-03-22T19:09:51.241586+0000","last_active":"2024-03-22T19:09:51.241586+0000","last_peered":"2024-03-22T19:09:51.241586+0000","last_clean":"2024-03-22T19:09:51.241586+0000","last_became_active":"2024-03-22T19:09:51.235327+0000","last_became_peered":"2024-03-22T19:09:51.235327+0000","last_unstale":"2024-03-22T19:09:51.241586+0000","last_undegraded":"2024-03-22T19:09:51.241586+0000","last_fullsized":"2024-03-22T19:09:51.241586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:43:59.307732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230257+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:50.230257+0000","last_peered":"2024-03-22T19:09:50.230257+0000","last_clean":"2024-03-22T19:09:50.230257+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:50.230257+0000","last_undegraded":"2024-03-22T19:09:50.230257+0000","last_fullsized":"2024-03-22T19:09:50.230257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231141+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:50.231141+0000","last_peered":"2024-03-22T19:09:50.231141+0000","last_clean":"2024-03-22T19:09:50.231141+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:50.231141+0000","last_undegraded":"2024-03-22T19:09:50.231141+0000","last_fullsized":"2024-03-22T19:09:50.231141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224983+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:50.224983+0000","last_peered":"2024-03-22T19:09:50.224983+0000","last_clean":"2024-03-22T19:09:50.224983+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:50.224983+0000","last_undegraded":"2024-03-22T19:09:50.224983+0000","last_fullsized":"2024-03-22T19:09:50.224983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":106,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:44.233437+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:10:44.233437+0000","last_peered":"2024-03-22T19:10:44.233437+0000","last_clean":"2024-03-22T19:10:44.233437+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:10:44.233437+0000","last_undegraded":"2024-03-22T19:10:44.233437+0000","last_fullsized":"2024-03-22T19:10:44.233437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224702+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:50.224702+0000","last_peered":"2024-03-22T19:09:50.224702+0000","last_clean":"2024-03-22T19:09:50.224702+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:50.224702+0000","last_undegraded":"2024-03-22T19:09:50.224702+0000","last_fullsized":"2024-03-22T19:09:50.224702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232307+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:50.232307+0000","last_peered":"2024-03-22T19:09:50.232307+0000","last_clean":"2024-03-22T19:09:50.232307+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:50.232307+0000","last_undegraded":"2024-03-22T19:09:50.232307+0000","last_fullsized":"2024-03-22T19:09:50.232307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221786+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:50.221786+0000","last_peered":"2024-03-22T19:09:50.221786+0000","last_clean":"2024-03-22T19:09:50.221786+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:50.221786+0000","last_undegraded":"2024-03-22T19:09:50.221786+0000","last_fullsized":"2024-03-22T19:09:50.221786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224337+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:50.224337+0000","last_peered":"2024-03-22T19:09:50.224337+0000","last_clean":"2024-03-22T19:09:50.224337+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:50.224337+0000","last_undegraded":"2024-03-22T19:09:50.224337+0000","last_fullsized":"2024-03-22T19:09:50.224337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229345+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:50.229345+0000","last_peered":"2024-03-22T19:09:50.229345+0000","last_clean":"2024-03-22T19:09:50.229345+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:50.229345+0000","last_undegraded":"2024-03-22T19:09:50.229345+0000","last_fullsized":"2024-03-22T19:09:50.229345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.412328+0000","last_change":"2024-03-22T19:09:51.412328+0000","last_active":"2024-03-22T19:09:51.412328+0000","last_peered":"2024-03-22T19:09:51.412328+0000","last_clean":"2024-03-22T19:09:51.412328+0000","last_became_active":"2024-03-22T19:09:51.411942+0000","last_became_peered":"2024-03-22T19:09:51.411942+0000","last_unstale":"2024-03-22T19:09:51.412328+0000","last_undegraded":"2024-03-22T19:09:51.412328+0000","last_fullsized":"2024-03-22T19:09:51.412328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:39:48.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225949+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:50.225949+0000","last_peered":"2024-03-22T19:09:50.225949+0000","last_clean":"2024-03-22T19:09:50.225949+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:50.225949+0000","last_undegraded":"2024-03-22T19:09:50.225949+0000","last_fullsized":"2024-03-22T19:09:50.225949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225646+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:50.225646+0000","last_peered":"2024-03-22T19:09:50.225646+0000","last_clean":"2024-03-22T19:09:50.225646+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:50.225646+0000","last_undegraded":"2024-03-22T19:09:50.225646+0000","last_fullsized":"2024-03-22T19:09:50.225646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280034+0000","last_change":"2024-03-22T19:09:51.280034+0000","last_active":"2024-03-22T19:09:51.280034+0000","last_peered":"2024-03-22T19:09:51.280034+0000","last_clean":"2024-03-22T19:09:51.280034+0000","last_became_active":"2024-03-22T19:09:51.279604+0000","last_became_peered":"2024-03-22T19:09:51.279604+0000","last_unstale":"2024-03-22T19:09:51.280034+0000","last_undegraded":"2024-03-22T19:09:51.280034+0000","last_fullsized":"2024-03-22T19:09:51.280034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:38:59.518086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229596+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:50.229596+0000","last_peered":"2024-03-22T19:09:50.229596+0000","last_clean":"2024-03-22T19:09:50.229596+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:50.229596+0000","last_undegraded":"2024-03-22T19:09:50.229596+0000","last_fullsized":"2024-03-22T19:09:50.229596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226600+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:50.226600+0000","last_peered":"2024-03-22T19:09:50.226600+0000","last_clean":"2024-03-22T19:09:50.226600+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:50.226600+0000","last_undegraded":"2024-03-22T19:09:50.226600+0000","last_fullsized":"2024-03-22T19:09:50.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230693+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:50.230693+0000","last_peered":"2024-03-22T19:09:50.230693+0000","last_clean":"2024-03-22T19:09:50.230693+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:50.230693+0000","last_undegraded":"2024-03-22T19:09:50.230693+0000","last_fullsized":"2024-03-22T19:09:50.230693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229186+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:50.229186+0000","last_peered":"2024-03-22T19:09:50.229186+0000","last_clean":"2024-03-22T19:09:50.229186+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:50.229186+0000","last_undegraded":"2024-03-22T19:09:50.229186+0000","last_fullsized":"2024-03-22T19:09:50.229186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230140+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:50.230140+0000","last_peered":"2024-03-22T19:09:50.230140+0000","last_clean":"2024-03-22T19:09:50.230140+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:50.230140+0000","last_undegraded":"2024-03-22T19:09:50.230140+0000","last_fullsized":"2024-03-22T19:09:50.230140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228785+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:50.228785+0000","last_peered":"2024-03-22T19:09:50.228785+0000","last_clean":"2024-03-22T19:09:50.228785+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:50.228785+0000","last_undegraded":"2024-03-22T19:09:50.228785+0000","last_fullsized":"2024-03-22T19:09:50.228785+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221650+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:50.221650+0000","last_peered":"2024-03-22T19:09:50.221650+0000","last_clean":"2024-03-22T19:09:50.221650+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:50.221650+0000","last_undegraded":"2024-03-22T19:09:50.221650+0000","last_fullsized":"2024-03-22T19:09:50.221650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229710+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:50.229710+0000","last_peered":"2024-03-22T19:09:50.229710+0000","last_clean":"2024-03-22T19:09:50.229710+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:50.229710+0000","last_undegraded":"2024-03-22T19:09:50.229710+0000","last_fullsized":"2024-03-22T19:09:50.229710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66581,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":890,"num_read_kb":519,"num_write":233,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199743,"data_compressed":5466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":430423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95299999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.498,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82299999999999995,"5min":1.109,"15min":1.109},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.54500000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.032,"5min":1.214,"15min":1.214},"last":0.752},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78600000000000003,"15min":0.79200000000000004},"min":{"1min":0.53600000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89000000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.63,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.73099999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.63800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":1.355,"15min":1.355},"last":0.74299999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1080662,"data_compressed":37475,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57099999999999995,"5min":0.628,"15min":0.628},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.58499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.624}]},{"osd":2,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.69099999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69099999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77200000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69799999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80600000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.749},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.875,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90000000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.58399999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86799999999999999,"5min":1.159,"15min":1.159},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.55800000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.81200000000000006}]},{"osd":11,"last update":"Fri Mar 22 19:10:40 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.63200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.61199999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82699999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":430289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.76500000000000001,"15min":0.78500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80200000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.70899999999999996,"15min":0.72099999999999997},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.048,"15min":1.048},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.998,"15min":0.998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.77000000000000002,"15min":0.80600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81299999999999994,"5min":1.355,"15min":1.355},"last":0.623}]},{"osd":5,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.87,"15min":0.89800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88200000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.504,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84199999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.89900000000000002,"15min":0.94199999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.86099999999999999,"15min":0.89300000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.87,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.84999999999999998,"15min":0.876},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96799999999999997,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.88200000000000001,"15min":0.90700000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0069999999999999,"5min":1.476,"15min":1.476},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.94699999999999995,"15min":0.98099999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98199999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.90100000000000002,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.86699999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.875,"15min":0.875},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.63500000000000001},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.60599999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.755,"15min":0.72499999999999998},"min":{"1min":0.66100000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.88600000000000001}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.56000000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Fri Mar 22 19:10:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.745},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.80000000000000004}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":1023067,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.438},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.35899999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69499999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.436,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.50800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83299999999999996,"5min":1.111,"15min":1.111},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.496,"15min":0.496},"min":{"1min":0.40899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.75700000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79100000000000004,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.56000000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88400000000000001,"5min":1.155,"15min":1.155},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.73599999999999999,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"last":0.57599999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:10:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.878,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.499},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.56299999999999994,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91000000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.626}]},{"osd":9,"last update":"Fri Mar 22 19:10:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92100000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.044,"15min":1.044},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.63700000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.60399999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73299999999999998}]},{"osd":11,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71699999999999997}]}]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":430264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0960000000000001,"5min":1.369,"15min":1.369},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.496},{"interface":"front","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.371,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.131,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.497,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.56000000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.73199999999999998}]},{"osd":8,"last update":"Fri Mar 22 19:09:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.82299999999999995,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.87,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.68600000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.77900000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.626},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Mar 22 19:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82899999999999996,"15min":0.82699999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":490521,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:40 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.7320000000000002,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.76100000000000001},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":6.71,"15min":6.71},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83699999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.379},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47999999999999998,"15min":0.442},"min":{"1min":0.35699999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.874,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86499999999999999,"5min":0.88,"15min":0.88},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Mar 22 19:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.441,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82399999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60199999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.77200000000000002,"15min":0.752},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.749},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67900000000000005}]},{"osd":10,"last update":"Fri Mar 22 19:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Mar 22 19:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":431297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.494,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.748}]},{"osd":3,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81499999999999995,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.63400000000000001},"min":{"1min":0.49099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64400000000000002,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:10:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.78200000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:10:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84999999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.67900000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1919999999999999,"5min":1.266,"15min":1.266},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.63700000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":489976,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.50600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.62}]},{"osd":1,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Mar 22 19:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70699999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.437},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65300000000000002,"5min":1.448,"15min":1.448},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.876,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.60299999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.69899999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.54800000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69699999999999995,"15min":0.68500000000000005},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.002},"last":0.80200000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72899999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84099999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.46200000000000002,"15min":0.432},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.56299999999999994},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2250000000000001,"5min":1.244,"15min":1.244},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0069999999999999,"5min":1.367,"15min":1.367},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84599999999999997,"5min":1.208,"15min":1.208},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.417,"15min":1.417},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97599999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.628}]},{"osd":6,"last update":"Fri Mar 22 19:10:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83099999999999996,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.069,"5min":1.641,"15min":1.641},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1279999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97899999999999998,"5min":1.163,"15min":1.163},"last":0.70199999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":0.87},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0800000000000001,"5min":1.355,"15min":1.355},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91500000000000004,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1639999999999999,"5min":1.331,"15min":1.331},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0229999999999999,"5min":1.302,"15min":1.302},"last":0.76300000000000001}]},{"osd":11,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.33800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58299999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.38300000000000001},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.56200000000000006,"5min":0.629,"15min":0.629},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.45300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80900000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.379},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.44,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.434}]},{"osd":5,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83599999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.65600000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.504}]},{"osd":8,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60099999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":1.214,"15min":1.214},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93899999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.01,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.65200000000000002}]},{"osd":11,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70899999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1022485,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67600000000000005,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85799999999999998,"5min":3.887,"15min":3.887},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.66200000000000003,"15min":0.71299999999999997},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38,"15min":0.37},"min":{"1min":0.32800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.371},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.497}]},{"osd":6,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.72899999999999998,"15min":0.77500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.749,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.621},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.751},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.45100000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.51200000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61663,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61809,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61341,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:10:48.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 -- 172.21.15.86:0/1344191533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 msgr2=0x7f82fc101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 --2- 172.21.15.86:0/1344191533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc101070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009920 tx=0x7f82ec009c30 comp rx=0 tx=0).stop 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 -- 172.21.15.86:0/1344191533 shutdown_connections 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 --2- 172.21.15.86:0/1344191533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 0x7f82fc109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 --2- 172.21.15.86:0/1344191533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc101070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 --2- 172.21.15.86:0/1344191533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc101e40 0x7f82fc1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 -- 172.21.15.86:0/1344191533 >> 172.21.15.86:0/1344191533 conn(0x7f82fc0fc1c0 msgr2=0x7f82fc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 -- 172.21.15.86:0/1344191533 shutdown_connections 2024-03-22T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.051+0000 7f8301e00700 1 -- 172.21.15.86:0/1344191533 wait complete. 2024-03-22T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 Processor -- start 2024-03-22T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 -- start start 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc101e40 0x7f82fc1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 0x7f82fc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a09a0 con 0x7f82fc1027e0 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a0ae0 con 0x7f82fc100c50 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f8301e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a0c70 con 0x7f82fc101e40 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f82faa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f82faa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53000/0 (socket says 172.21.15.86:53000) 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f82faa00700 1 -- 172.21.15.86:0/2399780073 learned_addr learned my addr 172.21.15.86:0/2399780073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.052+0000 7f82fb400700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 0x7f82fc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82fa000700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc101e40 0x7f82fc1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82faa00700 1 -- 172.21.15.86:0/2399780073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc101e40 msgr2=0x7f82fc1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82faa00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc101e40 0x7f82fc1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82faa00700 1 -- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 msgr2=0x7f82fc1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82faa00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 0x7f82fc1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82faa00700 1 -- 172.21.15.86:0/2399780073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4009d00 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82fb400700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 0x7f82fc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f82faa00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc198e70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009230 tx=0x7f82ec00bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec01c070 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82ec005040 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec009520 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004a30 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.053+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1a1180 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.054+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc193460 con 0x7f82fc100c50 2024-03-22T19:10:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.056+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f82ec020070 con 0x7f82fc100c50 2024-03-22T19:10:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.058+0000 7f8300a00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82e00a0b20 0x7f82e00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.058+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f82ec0cd330 con 0x7f82fc100c50 2024-03-22T19:10:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.058+0000 7f82fa000700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82e00a0b20 0x7f82e00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.058+0000 7f82fa000700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82e00a0b20 0x7f82e00a2fe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009bf0 tx=0x7f82e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.060+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f82ec096c20 con 0x7f82fc100c50 2024-03-22T19:10:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.270+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82fc0611f0 con 0x7f82e00a0b20 2024-03-22T19:10:49.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.281+0000 7f8300a00700 1 -- 172.21.15.86:0/2399780073 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344286 (secure 0 0 0) 0x7f82fc0611f0 con 0x7f82e00a0b20 2024-03-22T19:10:49.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82e00a0b20 msgr2=0x7f82e00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82e00a0b20 0x7f82e00a2fe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009bf0 tx=0x7f82e8009260 comp rx=0 tx=0).stop 2024-03-22T19:10:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 msgr2=0x7f82fc198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc198e70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009230 tx=0x7f82ec00bfa0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 shutdown_connections 2024-03-22T19:10:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82e00a0b20 0x7f82e00a2fe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1027e0 0x7f82fc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc100c50 0x7f82fc198e70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 --2- 172.21.15.86:0/2399780073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc101e40 0x7f82fc1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 >> 172.21.15.86:0/2399780073 conn(0x7f82fc0fc1c0 msgr2=0x7f82fc107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 shutdown_connections 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.286+0000 7f8301e00700 1 -- 172.21.15.86:0/2399780073 wait complete. 2024-03-22T19:10:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:10:49.307 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-03-22T19:10:48.172559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657183,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1016,"num_read_kb":628,"num_write":416,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335240,"kb_used_data":11000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606584,"statfs":{"total":1151940427776,"available":1151597142016,"internally_reserved":0,"allocated":11264000,"data_stored":7118970,"data_compressed":112425,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":612,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":64,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003269"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229812+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:50.229812+0000","last_peered":"2024-03-22T19:09:50.229812+0000","last_clean":"2024-03-22T19:09:50.229812+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:50.229812+0000","last_undegraded":"2024-03-22T19:09:50.229812+0000","last_fullsized":"2024-03-22T19:09:50.229812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225850+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:50.225850+0000","last_peered":"2024-03-22T19:09:50.225850+0000","last_clean":"2024-03-22T19:09:50.225850+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:50.225850+0000","last_undegraded":"2024-03-22T19:09:50.225850+0000","last_fullsized":"2024-03-22T19:09:50.225850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225832+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:50.225832+0000","last_peered":"2024-03-22T19:09:50.225832+0000","last_clean":"2024-03-22T19:09:50.225832+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:50.225832+0000","last_undegraded":"2024-03-22T19:09:50.225832+0000","last_fullsized":"2024-03-22T19:09:50.225832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223923+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:50.223923+0000","last_peered":"2024-03-22T19:09:50.223923+0000","last_clean":"2024-03-22T19:09:50.223923+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:50.223923+0000","last_undegraded":"2024-03-22T19:09:50.223923+0000","last_fullsized":"2024-03-22T19:09:50.223923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231033+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:50.231033+0000","last_peered":"2024-03-22T19:09:50.231033+0000","last_clean":"2024-03-22T19:09:50.231033+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:50.231033+0000","last_undegraded":"2024-03-22T19:09:50.231033+0000","last_fullsized":"2024-03-22T19:09:50.231033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227105+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:50.227105+0000","last_peered":"2024-03-22T19:09:50.227105+0000","last_clean":"2024-03-22T19:09:50.227105+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:50.227105+0000","last_undegraded":"2024-03-22T19:09:50.227105+0000","last_fullsized":"2024-03-22T19:09:50.227105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224692+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:50.224692+0000","last_peered":"2024-03-22T19:09:50.224692+0000","last_clean":"2024-03-22T19:09:50.224692+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:50.224692+0000","last_undegraded":"2024-03-22T19:09:50.224692+0000","last_fullsized":"2024-03-22T19:09:50.224692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230105+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:50.230105+0000","last_peered":"2024-03-22T19:09:50.230105+0000","last_clean":"2024-03-22T19:09:50.230105+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:50.230105+0000","last_undegraded":"2024-03-22T19:09:50.230105+0000","last_fullsized":"2024-03-22T19:09:50.230105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228937+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:50.228937+0000","last_peered":"2024-03-22T19:09:50.228937+0000","last_clean":"2024-03-22T19:09:50.228937+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:50.228937+0000","last_undegraded":"2024-03-22T19:09:50.228937+0000","last_fullsized":"2024-03-22T19:09:50.228937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280115+0000","last_change":"2024-03-22T19:09:51.280115+0000","last_active":"2024-03-22T19:09:51.280115+0000","last_peered":"2024-03-22T19:09:51.280115+0000","last_clean":"2024-03-22T19:09:51.280115+0000","last_became_active":"2024-03-22T19:09:51.279764+0000","last_became_peered":"2024-03-22T19:09:51.279764+0000","last_unstale":"2024-03-22T19:09:51.280115+0000","last_undegraded":"2024-03-22T19:09:51.280115+0000","last_fullsized":"2024-03-22T19:09:51.280115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:51:35.601327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224581+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:50.224581+0000","last_peered":"2024-03-22T19:09:50.224581+0000","last_clean":"2024-03-22T19:09:50.224581+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:50.224581+0000","last_undegraded":"2024-03-22T19:09:50.224581+0000","last_fullsized":"2024-03-22T19:09:50.224581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223728+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:50.223728+0000","last_peered":"2024-03-22T19:09:50.223728+0000","last_clean":"2024-03-22T19:09:50.223728+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:50.223728+0000","last_undegraded":"2024-03-22T19:09:50.223728+0000","last_fullsized":"2024-03-22T19:09:50.223728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221704+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:50.221704+0000","last_peered":"2024-03-22T19:09:50.221704+0000","last_clean":"2024-03-22T19:09:50.221704+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:50.221704+0000","last_undegraded":"2024-03-22T19:09:50.221704+0000","last_fullsized":"2024-03-22T19:09:50.221704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224532+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:50.224532+0000","last_peered":"2024-03-22T19:09:50.224532+0000","last_clean":"2024-03-22T19:09:50.224532+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:50.224532+0000","last_undegraded":"2024-03-22T19:09:50.224532+0000","last_fullsized":"2024-03-22T19:09:50.224532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225345+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:50.225345+0000","last_peered":"2024-03-22T19:09:50.225345+0000","last_clean":"2024-03-22T19:09:50.225345+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:50.225345+0000","last_undegraded":"2024-03-22T19:09:50.225345+0000","last_fullsized":"2024-03-22T19:09:50.225345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228414+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:50.228414+0000","last_peered":"2024-03-22T19:09:50.228414+0000","last_clean":"2024-03-22T19:09:50.228414+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:50.228414+0000","last_undegraded":"2024-03-22T19:09:50.228414+0000","last_fullsized":"2024-03-22T19:09:50.228414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229938+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:50.229938+0000","last_peered":"2024-03-22T19:09:50.229938+0000","last_clean":"2024-03-22T19:09:50.229938+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:50.229938+0000","last_undegraded":"2024-03-22T19:09:50.229938+0000","last_fullsized":"2024-03-22T19:09:50.229938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224855+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:50.224855+0000","last_peered":"2024-03-22T19:09:50.224855+0000","last_clean":"2024-03-22T19:09:50.224855+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:50.224855+0000","last_undegraded":"2024-03-22T19:09:50.224855+0000","last_fullsized":"2024-03-22T19:09:50.224855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226114+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:50.226114+0000","last_peered":"2024-03-22T19:09:50.226114+0000","last_clean":"2024-03-22T19:09:50.226114+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:50.226114+0000","last_undegraded":"2024-03-22T19:09:50.226114+0000","last_fullsized":"2024-03-22T19:09:50.226114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:44.232970+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:10:44.232970+0000","last_peered":"2024-03-22T19:10:44.232970+0000","last_clean":"2024-03-22T19:10:44.232970+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:10:44.232970+0000","last_undegraded":"2024-03-22T19:10:44.232970+0000","last_fullsized":"2024-03-22T19:10:44.232970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230751+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:50.230751+0000","last_peered":"2024-03-22T19:09:50.230751+0000","last_clean":"2024-03-22T19:09:50.230751+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:50.230751+0000","last_undegraded":"2024-03-22T19:09:50.230751+0000","last_fullsized":"2024-03-22T19:09:50.230751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230850+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:50.230850+0000","last_peered":"2024-03-22T19:09:50.230850+0000","last_clean":"2024-03-22T19:09:50.230850+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:50.230850+0000","last_undegraded":"2024-03-22T19:09:50.230850+0000","last_fullsized":"2024-03-22T19:09:50.230850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226153+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:50.226153+0000","last_peered":"2024-03-22T19:09:50.226153+0000","last_clean":"2024-03-22T19:09:50.226153+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:50.226153+0000","last_undegraded":"2024-03-22T19:09:50.226153+0000","last_fullsized":"2024-03-22T19:09:50.226153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231344+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:50.231344+0000","last_peered":"2024-03-22T19:09:50.231344+0000","last_clean":"2024-03-22T19:09:50.231344+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:50.231344+0000","last_undegraded":"2024-03-22T19:09:50.231344+0000","last_fullsized":"2024-03-22T19:09:50.231344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228584+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:50.228584+0000","last_peered":"2024-03-22T19:09:50.228584+0000","last_clean":"2024-03-22T19:09:50.228584+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:50.228584+0000","last_undegraded":"2024-03-22T19:09:50.228584+0000","last_fullsized":"2024-03-22T19:09:50.228584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231109+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:50.231109+0000","last_peered":"2024-03-22T19:09:50.231109+0000","last_clean":"2024-03-22T19:09:50.231109+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:50.231109+0000","last_undegraded":"2024-03-22T19:09:50.231109+0000","last_fullsized":"2024-03-22T19:09:50.231109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232324+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:50.232324+0000","last_peered":"2024-03-22T19:09:50.232324+0000","last_clean":"2024-03-22T19:09:50.232324+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:50.232324+0000","last_undegraded":"2024-03-22T19:09:50.232324+0000","last_fullsized":"2024-03-22T19:09:50.232324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:45.234575+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:10:45.234575+0000","last_peered":"2024-03-22T19:10:45.234575+0000","last_clean":"2024-03-22T19:10:45.234575+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:10:45.234575+0000","last_undegraded":"2024-03-22T19:10:45.234575+0000","last_fullsized":"2024-03-22T19:10:45.234575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232010+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:50.232010+0000","last_peered":"2024-03-22T19:09:50.232010+0000","last_clean":"2024-03-22T19:09:50.232010+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:50.232010+0000","last_undegraded":"2024-03-22T19:09:50.232010+0000","last_fullsized":"2024-03-22T19:09:50.232010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227866+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:50.227866+0000","last_peered":"2024-03-22T19:09:50.227866+0000","last_clean":"2024-03-22T19:09:50.227866+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:50.227866+0000","last_undegraded":"2024-03-22T19:09:50.227866+0000","last_fullsized":"2024-03-22T19:09:50.227866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230805+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:50.230805+0000","last_peered":"2024-03-22T19:09:50.230805+0000","last_clean":"2024-03-22T19:09:50.230805+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:50.230805+0000","last_undegraded":"2024-03-22T19:09:50.230805+0000","last_fullsized":"2024-03-22T19:09:50.230805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221505+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:50.221505+0000","last_peered":"2024-03-22T19:09:50.221505+0000","last_clean":"2024-03-22T19:09:50.221505+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:50.221505+0000","last_undegraded":"2024-03-22T19:09:50.221505+0000","last_fullsized":"2024-03-22T19:09:50.221505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230067+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:50.230067+0000","last_peered":"2024-03-22T19:09:50.230067+0000","last_clean":"2024-03-22T19:09:50.230067+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:50.230067+0000","last_undegraded":"2024-03-22T19:09:50.230067+0000","last_fullsized":"2024-03-22T19:09:50.230067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229198+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:50.229198+0000","last_peered":"2024-03-22T19:09:50.229198+0000","last_clean":"2024-03-22T19:09:50.229198+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:50.229198+0000","last_undegraded":"2024-03-22T19:09:50.229198+0000","last_fullsized":"2024-03-22T19:09:50.229198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224276+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:50.224276+0000","last_peered":"2024-03-22T19:09:50.224276+0000","last_clean":"2024-03-22T19:09:50.224276+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:50.224276+0000","last_undegraded":"2024-03-22T19:09:50.224276+0000","last_fullsized":"2024-03-22T19:09:50.224276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":190,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:46.235367+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:10:46.235367+0000","last_peered":"2024-03-22T19:10:46.235367+0000","last_clean":"2024-03-22T19:10:46.235367+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:10:46.235367+0000","last_undegraded":"2024-03-22T19:10:46.235367+0000","last_fullsized":"2024-03-22T19:10:46.235367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228903+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:50.228903+0000","last_peered":"2024-03-22T19:09:50.228903+0000","last_clean":"2024-03-22T19:09:50.228903+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:50.228903+0000","last_undegraded":"2024-03-22T19:09:50.228903+0000","last_fullsized":"2024-03-22T19:09:50.228903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.438753+0000","last_change":"2024-03-22T19:09:51.438753+0000","last_active":"2024-03-22T19:09:51.438753+0000","last_peered":"2024-03-22T19:09:51.438753+0000","last_clean":"2024-03-22T19:09:51.438753+0000","last_became_active":"2024-03-22T19:09:51.413301+0000","last_became_peered":"2024-03-22T19:09:51.413301+0000","last_unstale":"2024-03-22T19:09:51.438753+0000","last_undegraded":"2024-03-22T19:09:51.438753+0000","last_fullsized":"2024-03-22T19:09:51.438753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:49.994282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226106+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:50.226106+0000","last_peered":"2024-03-22T19:09:50.226106+0000","last_clean":"2024-03-22T19:09:50.226106+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:50.226106+0000","last_undegraded":"2024-03-22T19:09:50.226106+0000","last_fullsized":"2024-03-22T19:09:50.226106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232420+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:50.232420+0000","last_peered":"2024-03-22T19:09:50.232420+0000","last_clean":"2024-03-22T19:09:50.232420+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:50.232420+0000","last_undegraded":"2024-03-22T19:09:50.232420+0000","last_fullsized":"2024-03-22T19:09:50.232420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228700+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:50.228700+0000","last_peered":"2024-03-22T19:09:50.228700+0000","last_clean":"2024-03-22T19:09:50.228700+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:50.228700+0000","last_undegraded":"2024-03-22T19:09:50.228700+0000","last_fullsized":"2024-03-22T19:09:50.228700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":223,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:46.250848+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:10:46.250848+0000","last_peered":"2024-03-22T19:10:46.250848+0000","last_clean":"2024-03-22T19:10:46.250848+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:10:46.250848+0000","last_undegraded":"2024-03-22T19:10:46.250848+0000","last_fullsized":"2024-03-22T19:10:46.250848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59653,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":148,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230888+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:50.230888+0000","last_peered":"2024-03-22T19:09:50.230888+0000","last_clean":"2024-03-22T19:09:50.230888+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:50.230888+0000","last_undegraded":"2024-03-22T19:09:50.230888+0000","last_fullsized":"2024-03-22T19:09:50.230888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226121+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:50.226121+0000","last_peered":"2024-03-22T19:09:50.226121+0000","last_clean":"2024-03-22T19:09:50.226121+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:50.226121+0000","last_undegraded":"2024-03-22T19:09:50.226121+0000","last_fullsized":"2024-03-22T19:09:50.226121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221747+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:50.221747+0000","last_peered":"2024-03-22T19:09:50.221747+0000","last_clean":"2024-03-22T19:09:50.221747+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:50.221747+0000","last_undegraded":"2024-03-22T19:09:50.221747+0000","last_fullsized":"2024-03-22T19:09:50.221747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230590+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:50.230590+0000","last_peered":"2024-03-22T19:09:50.230590+0000","last_clean":"2024-03-22T19:09:50.230590+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:50.230590+0000","last_undegraded":"2024-03-22T19:09:50.230590+0000","last_fullsized":"2024-03-22T19:09:50.230590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224672+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:50.224672+0000","last_peered":"2024-03-22T19:09:50.224672+0000","last_clean":"2024-03-22T19:09:50.224672+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:50.224672+0000","last_undegraded":"2024-03-22T19:09:50.224672+0000","last_fullsized":"2024-03-22T19:09:50.224672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:43.232065+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:10:43.232065+0000","last_peered":"2024-03-22T19:10:43.232065+0000","last_clean":"2024-03-22T19:10:43.232065+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:10:43.232065+0000","last_undegraded":"2024-03-22T19:10:43.232065+0000","last_fullsized":"2024-03-22T19:10:43.232065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.435708+0000","last_change":"2024-03-22T19:09:51.435708+0000","last_active":"2024-03-22T19:09:51.435708+0000","last_peered":"2024-03-22T19:09:51.435708+0000","last_clean":"2024-03-22T19:09:51.435708+0000","last_became_active":"2024-03-22T19:09:51.412550+0000","last_became_peered":"2024-03-22T19:09:51.412550+0000","last_unstale":"2024-03-22T19:09:51.435708+0000","last_undegraded":"2024-03-22T19:09:51.435708+0000","last_fullsized":"2024-03-22T19:09:51.435708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:11:44.092847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229832+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:50.229832+0000","last_peered":"2024-03-22T19:09:50.229832+0000","last_clean":"2024-03-22T19:09:50.229832+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:50.229832+0000","last_undegraded":"2024-03-22T19:09:50.229832+0000","last_fullsized":"2024-03-22T19:09:50.229832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231493+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:50.231493+0000","last_peered":"2024-03-22T19:09:50.231493+0000","last_clean":"2024-03-22T19:09:50.231493+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:50.231493+0000","last_undegraded":"2024-03-22T19:09:50.231493+0000","last_fullsized":"2024-03-22T19:09:50.231493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229589+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:50.229589+0000","last_peered":"2024-03-22T19:09:50.229589+0000","last_clean":"2024-03-22T19:09:50.229589+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:50.229589+0000","last_undegraded":"2024-03-22T19:09:50.229589+0000","last_fullsized":"2024-03-22T19:09:50.229589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225716+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:50.225716+0000","last_peered":"2024-03-22T19:09:50.225716+0000","last_clean":"2024-03-22T19:09:50.225716+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:50.225716+0000","last_undegraded":"2024-03-22T19:09:50.225716+0000","last_fullsized":"2024-03-22T19:09:50.225716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227355+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:50.227355+0000","last_peered":"2024-03-22T19:09:50.227355+0000","last_clean":"2024-03-22T19:09:50.227355+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:50.227355+0000","last_undegraded":"2024-03-22T19:09:50.227355+0000","last_fullsized":"2024-03-22T19:09:50.227355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:45.234724+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:10:45.234724+0000","last_peered":"2024-03-22T19:10:45.234724+0000","last_clean":"2024-03-22T19:10:45.234724+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:10:45.234724+0000","last_undegraded":"2024-03-22T19:10:45.234724+0000","last_fullsized":"2024-03-22T19:10:45.234724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227464+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:50.227464+0000","last_peered":"2024-03-22T19:09:50.227464+0000","last_clean":"2024-03-22T19:09:50.227464+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:50.227464+0000","last_undegraded":"2024-03-22T19:09:50.227464+0000","last_fullsized":"2024-03-22T19:09:50.227464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229382+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:50.229382+0000","last_peered":"2024-03-22T19:09:50.229382+0000","last_clean":"2024-03-22T19:09:50.229382+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:50.229382+0000","last_undegraded":"2024-03-22T19:09:50.229382+0000","last_fullsized":"2024-03-22T19:09:50.229382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:43.231386+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:10:43.231386+0000","last_peered":"2024-03-22T19:10:43.231386+0000","last_clean":"2024-03-22T19:10:43.231386+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:10:43.231386+0000","last_undegraded":"2024-03-22T19:10:43.231386+0000","last_fullsized":"2024-03-22T19:10:43.231386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230745+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:50.230745+0000","last_peered":"2024-03-22T19:09:50.230745+0000","last_clean":"2024-03-22T19:09:50.230745+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:50.230745+0000","last_undegraded":"2024-03-22T19:09:50.230745+0000","last_fullsized":"2024-03-22T19:09:50.230745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223407+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:50.223407+0000","last_peered":"2024-03-22T19:09:50.223407+0000","last_clean":"2024-03-22T19:09:50.223407+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:50.223407+0000","last_undegraded":"2024-03-22T19:09:50.223407+0000","last_fullsized":"2024-03-22T19:09:50.223407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224404+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:50.224404+0000","last_peered":"2024-03-22T19:09:50.224404+0000","last_clean":"2024-03-22T19:09:50.224404+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:50.224404+0000","last_undegraded":"2024-03-22T19:09:50.224404+0000","last_fullsized":"2024-03-22T19:09:50.224404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221409+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:50.221409+0000","last_peered":"2024-03-22T19:09:50.221409+0000","last_clean":"2024-03-22T19:09:50.221409+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:50.221409+0000","last_undegraded":"2024-03-22T19:09:50.221409+0000","last_fullsized":"2024-03-22T19:09:50.221409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224573+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:50.224573+0000","last_peered":"2024-03-22T19:09:50.224573+0000","last_clean":"2024-03-22T19:09:50.224573+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:50.224573+0000","last_undegraded":"2024-03-22T19:09:50.224573+0000","last_fullsized":"2024-03-22T19:09:50.224573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231242+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:50.231242+0000","last_peered":"2024-03-22T19:09:50.231242+0000","last_clean":"2024-03-22T19:09:50.231242+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:50.231242+0000","last_undegraded":"2024-03-22T19:09:50.231242+0000","last_fullsized":"2024-03-22T19:09:50.231242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221846+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:50.221846+0000","last_peered":"2024-03-22T19:09:50.221846+0000","last_clean":"2024-03-22T19:09:50.221846+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:50.221846+0000","last_undegraded":"2024-03-22T19:09:50.221846+0000","last_fullsized":"2024-03-22T19:09:50.221846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224775+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:50.224775+0000","last_peered":"2024-03-22T19:09:50.224775+0000","last_clean":"2024-03-22T19:09:50.224775+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:50.224775+0000","last_undegraded":"2024-03-22T19:09:50.224775+0000","last_fullsized":"2024-03-22T19:09:50.224775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230722+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:50.230722+0000","last_peered":"2024-03-22T19:09:50.230722+0000","last_clean":"2024-03-22T19:09:50.230722+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:50.230722+0000","last_undegraded":"2024-03-22T19:09:50.230722+0000","last_fullsized":"2024-03-22T19:09:50.230722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223816+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:50.223816+0000","last_peered":"2024-03-22T19:09:50.223816+0000","last_clean":"2024-03-22T19:09:50.223816+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:50.223816+0000","last_undegraded":"2024-03-22T19:09:50.223816+0000","last_fullsized":"2024-03-22T19:09:50.223816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225947+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:50.225947+0000","last_peered":"2024-03-22T19:09:50.225947+0000","last_clean":"2024-03-22T19:09:50.225947+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:50.225947+0000","last_undegraded":"2024-03-22T19:09:50.225947+0000","last_fullsized":"2024-03-22T19:09:50.225947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228471+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:50.228471+0000","last_peered":"2024-03-22T19:09:50.228471+0000","last_clean":"2024-03-22T19:09:50.228471+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:50.228471+0000","last_undegraded":"2024-03-22T19:09:50.228471+0000","last_fullsized":"2024-03-22T19:09:50.228471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230778+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:50.230778+0000","last_peered":"2024-03-22T19:09:50.230778+0000","last_clean":"2024-03-22T19:09:50.230778+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:50.230778+0000","last_undegraded":"2024-03-22T19:09:50.230778+0000","last_fullsized":"2024-03-22T19:09:50.230778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230201+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:50.230201+0000","last_peered":"2024-03-22T19:09:50.230201+0000","last_clean":"2024-03-22T19:09:50.230201+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:50.230201+0000","last_undegraded":"2024-03-22T19:09:50.230201+0000","last_fullsized":"2024-03-22T19:09:50.230201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229252+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:50.229252+0000","last_peered":"2024-03-22T19:09:50.229252+0000","last_clean":"2024-03-22T19:09:50.229252+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:50.229252+0000","last_undegraded":"2024-03-22T19:09:50.229252+0000","last_fullsized":"2024-03-22T19:09:50.229252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.241586+0000","last_change":"2024-03-22T19:09:51.241586+0000","last_active":"2024-03-22T19:09:51.241586+0000","last_peered":"2024-03-22T19:09:51.241586+0000","last_clean":"2024-03-22T19:09:51.241586+0000","last_became_active":"2024-03-22T19:09:51.235327+0000","last_became_peered":"2024-03-22T19:09:51.235327+0000","last_unstale":"2024-03-22T19:09:51.241586+0000","last_undegraded":"2024-03-22T19:09:51.241586+0000","last_fullsized":"2024-03-22T19:09:51.241586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:43:59.307732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230257+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:50.230257+0000","last_peered":"2024-03-22T19:09:50.230257+0000","last_clean":"2024-03-22T19:09:50.230257+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:50.230257+0000","last_undegraded":"2024-03-22T19:09:50.230257+0000","last_fullsized":"2024-03-22T19:09:50.230257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231141+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:50.231141+0000","last_peered":"2024-03-22T19:09:50.231141+0000","last_clean":"2024-03-22T19:09:50.231141+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:50.231141+0000","last_undegraded":"2024-03-22T19:09:50.231141+0000","last_fullsized":"2024-03-22T19:09:50.231141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224983+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:50.224983+0000","last_peered":"2024-03-22T19:09:50.224983+0000","last_clean":"2024-03-22T19:09:50.224983+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:50.224983+0000","last_undegraded":"2024-03-22T19:09:50.224983+0000","last_fullsized":"2024-03-22T19:09:50.224983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":106,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:44.233437+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:10:44.233437+0000","last_peered":"2024-03-22T19:10:44.233437+0000","last_clean":"2024-03-22T19:10:44.233437+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:10:44.233437+0000","last_undegraded":"2024-03-22T19:10:44.233437+0000","last_fullsized":"2024-03-22T19:10:44.233437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224702+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:50.224702+0000","last_peered":"2024-03-22T19:09:50.224702+0000","last_clean":"2024-03-22T19:09:50.224702+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:50.224702+0000","last_undegraded":"2024-03-22T19:09:50.224702+0000","last_fullsized":"2024-03-22T19:09:50.224702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232307+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:50.232307+0000","last_peered":"2024-03-22T19:09:50.232307+0000","last_clean":"2024-03-22T19:09:50.232307+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:50.232307+0000","last_undegraded":"2024-03-22T19:09:50.232307+0000","last_fullsized":"2024-03-22T19:09:50.232307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221786+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:50.221786+0000","last_peered":"2024-03-22T19:09:50.221786+0000","last_clean":"2024-03-22T19:09:50.221786+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:50.221786+0000","last_undegraded":"2024-03-22T19:09:50.221786+0000","last_fullsized":"2024-03-22T19:09:50.221786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224337+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:50.224337+0000","last_peered":"2024-03-22T19:09:50.224337+0000","last_clean":"2024-03-22T19:09:50.224337+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:50.224337+0000","last_undegraded":"2024-03-22T19:09:50.224337+0000","last_fullsized":"2024-03-22T19:09:50.224337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229345+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:50.229345+0000","last_peered":"2024-03-22T19:09:50.229345+0000","last_clean":"2024-03-22T19:09:50.229345+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:50.229345+0000","last_undegraded":"2024-03-22T19:09:50.229345+0000","last_fullsized":"2024-03-22T19:09:50.229345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.412328+0000","last_change":"2024-03-22T19:09:51.412328+0000","last_active":"2024-03-22T19:09:51.412328+0000","last_peered":"2024-03-22T19:09:51.412328+0000","last_clean":"2024-03-22T19:09:51.412328+0000","last_became_active":"2024-03-22T19:09:51.411942+0000","last_became_peered":"2024-03-22T19:09:51.411942+0000","last_unstale":"2024-03-22T19:09:51.412328+0000","last_undegraded":"2024-03-22T19:09:51.412328+0000","last_fullsized":"2024-03-22T19:09:51.412328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:39:48.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225949+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:50.225949+0000","last_peered":"2024-03-22T19:09:50.225949+0000","last_clean":"2024-03-22T19:09:50.225949+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:50.225949+0000","last_undegraded":"2024-03-22T19:09:50.225949+0000","last_fullsized":"2024-03-22T19:09:50.225949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225646+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:50.225646+0000","last_peered":"2024-03-22T19:09:50.225646+0000","last_clean":"2024-03-22T19:09:50.225646+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:50.225646+0000","last_undegraded":"2024-03-22T19:09:50.225646+0000","last_fullsized":"2024-03-22T19:09:50.225646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280034+0000","last_change":"2024-03-22T19:09:51.280034+0000","last_active":"2024-03-22T19:09:51.280034+0000","last_peered":"2024-03-22T19:09:51.280034+0000","last_clean":"2024-03-22T19:09:51.280034+0000","last_became_active":"2024-03-22T19:09:51.279604+0000","last_became_peered":"2024-03-22T19:09:51.279604+0000","last_unstale":"2024-03-22T19:09:51.280034+0000","last_undegraded":"2024-03-22T19:09:51.280034+0000","last_fullsized":"2024-03-22T19:09:51.280034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:38:59.518086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229596+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:50.229596+0000","last_peered":"2024-03-22T19:09:50.229596+0000","last_clean":"2024-03-22T19:09:50.229596+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:50.229596+0000","last_undegraded":"2024-03-22T19:09:50.229596+0000","last_fullsized":"2024-03-22T19:09:50.229596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226600+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:50.226600+0000","last_peered":"2024-03-22T19:09:50.226600+0000","last_clean":"2024-03-22T19:09:50.226600+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:50.226600+0000","last_undegraded":"2024-03-22T19:09:50.226600+0000","last_fullsized":"2024-03-22T19:09:50.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230693+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:50.230693+0000","last_peered":"2024-03-22T19:09:50.230693+0000","last_clean":"2024-03-22T19:09:50.230693+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:50.230693+0000","last_undegraded":"2024-03-22T19:09:50.230693+0000","last_fullsized":"2024-03-22T19:09:50.230693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229186+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:50.229186+0000","last_peered":"2024-03-22T19:09:50.229186+0000","last_clean":"2024-03-22T19:09:50.229186+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:50.229186+0000","last_undegraded":"2024-03-22T19:09:50.229186+0000","last_fullsized":"2024-03-22T19:09:50.229186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230140+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:50.230140+0000","last_peered":"2024-03-22T19:09:50.230140+0000","last_clean":"2024-03-22T19:09:50.230140+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:50.230140+0000","last_undegraded":"2024-03-22T19:09:50.230140+0000","last_fullsized":"2024-03-22T19:09:50.230140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228785+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:50.228785+0000","last_peered":"2024-03-22T19:09:50.228785+0000","last_clean":"2024-03-22T19:09:50.228785+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:50.228785+0000","last_undegraded":"2024-03-22T19:09:50.228785+0000","last_fullsized":"2024-03-22T19:09:50.228785+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221650+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:50.221650+0000","last_peered":"2024-03-22T19:09:50.221650+0000","last_clean":"2024-03-22T19:09:50.221650+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:50.221650+0000","last_undegraded":"2024-03-22T19:09:50.221650+0000","last_fullsized":"2024-03-22T19:09:50.221650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229710+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:50.229710+0000","last_peered":"2024-03-22T19:09:50.229710+0000","last_clean":"2024-03-22T19:09:50.229710+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:50.229710+0000","last_undegraded":"2024-03-22T19:09:50.229710+0000","last_fullsized":"2024-03-22T19:09:50.229710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66581,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":890,"num_read_kb":519,"num_write":233,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199743,"data_compressed":5466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":430423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95299999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.498,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82299999999999995,"5min":1.109,"15min":1.109},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.54500000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.032,"5min":1.214,"15min":1.214},"last":0.752},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78600000000000003,"15min":0.79200000000000004},"min":{"1min":0.53600000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89000000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.63,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.73099999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.63800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":1.355,"15min":1.355},"last":0.74299999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1080662,"data_compressed":37475,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57099999999999995,"5min":0.628,"15min":0.628},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.58499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.624}]},{"osd":2,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.69099999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69099999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77200000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69799999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80600000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.749},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.875,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90000000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.58399999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86799999999999999,"5min":1.159,"15min":1.159},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.55800000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.81200000000000006}]},{"osd":11,"last update":"Fri Mar 22 19:10:40 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.63200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.61199999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82699999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":430289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.76500000000000001,"15min":0.78500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80200000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.70899999999999996,"15min":0.72099999999999997},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.048,"15min":1.048},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.998,"15min":0.998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.77000000000000002,"15min":0.80600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81299999999999994,"5min":1.355,"15min":1.355},"last":0.623}]},{"osd":5,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.87,"15min":0.89800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88200000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.504,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84199999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.89900000000000002,"15min":0.94199999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.86099999999999999,"15min":0.89300000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.87,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.84999999999999998,"15min":0.876},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96799999999999997,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.88200000000000001,"15min":0.90700000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0069999999999999,"5min":1.476,"15min":1.476},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.94699999999999995,"15min":0.98099999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98199999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.90100000000000002,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.86699999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.875,"15min":0.875},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.63500000000000001},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.60599999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.755,"15min":0.72499999999999998},"min":{"1min":0.66100000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.88600000000000001}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.56000000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Fri Mar 22 19:10:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.745},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.80000000000000004}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":1023067,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.438},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.35899999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69499999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.436,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.50800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83299999999999996,"5min":1.111,"15min":1.111},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.496,"15min":0.496},"min":{"1min":0.40899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.75700000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79100000000000004,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.56000000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88400000000000001,"5min":1.155,"15min":1.155},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.73599999999999999,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"last":0.57599999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:10:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.878,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.499},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.56299999999999994,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91000000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.626}]},{"osd":9,"last update":"Fri Mar 22 19:10:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92100000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.044,"15min":1.044},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.63700000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.60399999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73299999999999998}]},{"osd":11,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71699999999999997}]}]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":430264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0960000000000001,"5min":1.369,"15min":1.369},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.496},{"interface":"front","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.371,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.131,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.497,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.56000000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.73199999999999998}]},{"osd":8,"last update":"Fri Mar 22 19:09:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.82299999999999995,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.87,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.68600000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.77900000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.626},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Mar 22 19:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82899999999999996,"15min":0.82699999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":490521,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:40 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.7320000000000002,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.76100000000000001},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":6.71,"15min":6.71},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83699999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.379},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47999999999999998,"15min":0.442},"min":{"1min":0.35699999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.874,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86499999999999999,"5min":0.88,"15min":0.88},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Mar 22 19:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.441,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82399999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60199999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.77200000000000002,"15min":0.752},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.749},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67900000000000005}]},{"osd":10,"last update":"Fri Mar 22 19:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Mar 22 19:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":431297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.494,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.748}]},{"osd":3,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81499999999999995,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.63400000000000001},"min":{"1min":0.49099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64400000000000002,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:10:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.78200000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:10:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84999999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.67900000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1919999999999999,"5min":1.266,"15min":1.266},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.63700000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":489976,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.50600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.62}]},{"osd":1,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Mar 22 19:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70699999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.437},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65300000000000002,"5min":1.448,"15min":1.448},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.876,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.60299999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.69899999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.54800000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69699999999999995,"15min":0.68500000000000005},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.002},"last":0.80200000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72899999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84099999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.46200000000000002,"15min":0.432},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.56299999999999994},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2250000000000001,"5min":1.244,"15min":1.244},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0069999999999999,"5min":1.367,"15min":1.367},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84599999999999997,"5min":1.208,"15min":1.208},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.417,"15min":1.417},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97599999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.628}]},{"osd":6,"last update":"Fri Mar 22 19:10:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83099999999999996,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.069,"5min":1.641,"15min":1.641},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1279999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97899999999999998,"5min":1.163,"15min":1.163},"last":0.70199999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":0.87},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0800000000000001,"5min":1.355,"15min":1.355},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91500000000000004,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1639999999999999,"5min":1.331,"15min":1.331},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0229999999999999,"5min":1.302,"15min":1.302},"last":0.76300000000000001}]},{"osd":11,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.33800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58299999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.38300000000000001},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.56200000000000006,"5min":0.629,"15min":0.629},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.45300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80900000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.379},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.44,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.434}]},{"osd":5,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83599999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.65600000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.504}]},{"osd":8,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60099999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":1.214,"15min":1.214},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93899999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.01,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.65200000000000002}]},{"osd":11,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70899999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1022485,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67600000000000005,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85799999999999998,"5min":3.887,"15min":3.887},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.66200000000000003,"15min":0.71299999999999997},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38,"15min":0.37},"min":{"1min":0.32800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.371},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.497}]},{"osd":6,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.72899999999999998,"15min":0.77500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.749,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.621},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.751},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.45100000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.51200000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61663,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61809,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61341,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:10:49.312 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-22T19:10:49.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T19:10:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 -- 172.21.15.86:0/2361884646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c00737b0 msgr2=0x7f32c0073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 --2- 172.21.15.86:0/2361884646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c00737b0 0x7f32c0073c30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009990 tx=0x7f32ac009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 -- 172.21.15.86:0/2361884646 shutdown_connections 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 --2- 172.21.15.86:0/2361884646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 0x7f32c010af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 --2- 172.21.15.86:0/2361884646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32c0074d90 0x7f32c00731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 --2- 172.21.15.86:0/2361884646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c00737b0 0x7f32c0073c30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 -- 172.21.15.86:0/2361884646 >> 172.21.15.86:0/2361884646 conn(0x7f32c00fc1c0 msgr2=0x7f32c00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 -- 172.21.15.86:0/2361884646 shutdown_connections 2024-03-22T19:10:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.538+0000 7f32c6800700 1 -- 172.21.15.86:0/2361884646 wait complete. 2024-03-22T19:10:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.539+0000 7f32c6800700 1 Processor -- start 2024-03-22T19:10:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.539+0000 7f32c6800700 1 -- start start 2024-03-22T19:10:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.539+0000 7f32c6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32c00737b0 0x7f32c0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.539+0000 7f32c6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 0x7f32c0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.539+0000 7f32c6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 0x7f32c01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32c6800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c01a08f0 con 0x7f32c0074d90 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32c6800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c01a0a60 con 0x7f32c0108b20 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32c6800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c01a0c20 con 0x7f32c00737b0 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 0x7f32c0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 0x7f32c0199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42988/0 (socket says 172.21.15.86:42988) 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 -- 172.21.15.86:0/799462595 learned_addr learned my addr 172.21.15.86:0/799462595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bfe00700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 0x7f32c01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bf400700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32c00737b0 0x7f32c0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 -- 172.21.15.86:0/799462595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32c00737b0 msgr2=0x7f32c0198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32c00737b0 0x7f32c0198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 -- 172.21.15.86:0/799462595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 msgr2=0x7f32c01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 0x7f32c01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 -- 172.21.15.86:0/799462595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0095f0 con 0x7f32c0074d90 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bfe00700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 0x7f32c01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.540+0000 7f32bea00700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 0x7f32c0199340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32ac005910 tx=0x7f32ac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.541+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac01d070 con 0x7f32c0074d90 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.541+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32ac021410 con 0x7f32c0074d90 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.541+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c01a0ea0 con 0x7f32c0074d90 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.541+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac005720 con 0x7f32c0074d90 2024-03-22T19:10:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.541+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c01a1210 con 0x7f32c0074d90 2024-03-22T19:10:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.542+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c0107930 con 0x7f32c0074d90 2024-03-22T19:10:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.544+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f32ac021580 con 0x7f32c0074d90 2024-03-22T19:10:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.545+0000 7f32b7e00700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a40a0b20 0x7f32a40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.545+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f32ac0cd550 con 0x7f32c0074d90 2024-03-22T19:10:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.545+0000 7f32bf400700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a40a0b20 0x7f32a40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.546+0000 7f32bf400700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a40a0b20 0x7f32a40a2fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f32c019a280 tx=0x7f32b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.549+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f32ac096e40 con 0x7f32c0074d90 2024-03-22T19:10:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:49 smithi167 ceph-mon[47728]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-22T19:10:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:49 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1434816244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:49.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:49 smithi131 ceph-mon[78559]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-22T19:10:49.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:49 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1434816244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:49.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:49 smithi086 ceph-mon[38032]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-22T19:10:49.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:49 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1434816244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:49.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.761+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 --> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32c00611f0 con 0x7f32a40a0b20 2024-03-22T19:10:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.771+0000 7f32b7e00700 1 -- 172.21.15.86:0/799462595 <== mgr.14152 v2:172.21.15.86:6800/1126645338 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344286 (secure 0 0 0) 0x7f32c00611f0 con 0x7f32a40a0b20 2024-03-22T19:10:49.777 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a40a0b20 msgr2=0x7f32a40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a40a0b20 0x7f32a40a2fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f32c019a280 tx=0x7f32b0009260 comp rx=0 tx=0).stop 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 msgr2=0x7f32c0199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 0x7f32c0199340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32ac005910 tx=0x7f32ac009580 comp rx=0 tx=0).stop 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 shutdown_connections 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f32a40a0b20 0x7f32a40a2fe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c0108b20 0x7f32c01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32c00737b0 0x7f32c0198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 --2- 172.21.15.86:0/799462595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0074d90 0x7f32c0199340 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:49.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 >> 172.21.15.86:0/799462595 conn(0x7f32c00fc1c0 msgr2=0x7f32c0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:49.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.776+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 shutdown_connections 2024-03-22T19:10:49.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:49.777+0000 7f32c6800700 1 -- 172.21.15.86:0/799462595 wait complete. 2024-03-22T19:10:49.781 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T19:10:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-03-22T19:10:48.172559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657183,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1016,"num_read_kb":628,"num_write":416,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335240,"kb_used_data":11000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606584,"statfs":{"total":1151940427776,"available":1151597142016,"internally_reserved":0,"allocated":11264000,"data_stored":7118970,"data_compressed":112425,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":612,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":64,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003269"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229812+0000","last_change":"2024-03-22T19:09:13.643937+0000","last_active":"2024-03-22T19:09:50.229812+0000","last_peered":"2024-03-22T19:09:50.229812+0000","last_clean":"2024-03-22T19:09:50.229812+0000","last_became_active":"2024-03-22T19:09:13.643520+0000","last_became_peered":"2024-03-22T19:09:13.643520+0000","last_unstale":"2024-03-22T19:09:50.229812+0000","last_undegraded":"2024-03-22T19:09:50.229812+0000","last_fullsized":"2024-03-22T19:09:50.229812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:24:34.374740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225850+0000","last_change":"2024-03-22T19:09:13.638935+0000","last_active":"2024-03-22T19:09:50.225850+0000","last_peered":"2024-03-22T19:09:50.225850+0000","last_clean":"2024-03-22T19:09:50.225850+0000","last_became_active":"2024-03-22T19:09:13.636727+0000","last_became_peered":"2024-03-22T19:09:13.636727+0000","last_unstale":"2024-03-22T19:09:50.225850+0000","last_undegraded":"2024-03-22T19:09:50.225850+0000","last_fullsized":"2024-03-22T19:09:50.225850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:22:23.370550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225832+0000","last_change":"2024-03-22T19:09:13.646091+0000","last_active":"2024-03-22T19:09:50.225832+0000","last_peered":"2024-03-22T19:09:50.225832+0000","last_clean":"2024-03-22T19:09:50.225832+0000","last_became_active":"2024-03-22T19:09:13.645799+0000","last_became_peered":"2024-03-22T19:09:13.645799+0000","last_unstale":"2024-03-22T19:09:50.225832+0000","last_undegraded":"2024-03-22T19:09:50.225832+0000","last_fullsized":"2024-03-22T19:09:50.225832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:38:49.847280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223923+0000","last_change":"2024-03-22T19:09:13.646375+0000","last_active":"2024-03-22T19:09:50.223923+0000","last_peered":"2024-03-22T19:09:50.223923+0000","last_clean":"2024-03-22T19:09:50.223923+0000","last_became_active":"2024-03-22T19:09:13.646023+0000","last_became_peered":"2024-03-22T19:09:13.646023+0000","last_unstale":"2024-03-22T19:09:50.223923+0000","last_undegraded":"2024-03-22T19:09:50.223923+0000","last_fullsized":"2024-03-22T19:09:50.223923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:45:17.723478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231033+0000","last_change":"2024-03-22T19:09:13.642100+0000","last_active":"2024-03-22T19:09:50.231033+0000","last_peered":"2024-03-22T19:09:50.231033+0000","last_clean":"2024-03-22T19:09:50.231033+0000","last_became_active":"2024-03-22T19:09:13.641614+0000","last_became_peered":"2024-03-22T19:09:13.641614+0000","last_unstale":"2024-03-22T19:09:50.231033+0000","last_undegraded":"2024-03-22T19:09:50.231033+0000","last_fullsized":"2024-03-22T19:09:50.231033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:45:07.345049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227105+0000","last_change":"2024-03-22T19:09:13.646534+0000","last_active":"2024-03-22T19:09:50.227105+0000","last_peered":"2024-03-22T19:09:50.227105+0000","last_clean":"2024-03-22T19:09:50.227105+0000","last_became_active":"2024-03-22T19:09:13.646216+0000","last_became_peered":"2024-03-22T19:09:13.646216+0000","last_unstale":"2024-03-22T19:09:50.227105+0000","last_undegraded":"2024-03-22T19:09:50.227105+0000","last_fullsized":"2024-03-22T19:09:50.227105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:44:52.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224692+0000","last_change":"2024-03-22T19:09:13.631668+0000","last_active":"2024-03-22T19:09:50.224692+0000","last_peered":"2024-03-22T19:09:50.224692+0000","last_clean":"2024-03-22T19:09:50.224692+0000","last_became_active":"2024-03-22T19:09:13.631272+0000","last_became_peered":"2024-03-22T19:09:13.631272+0000","last_unstale":"2024-03-22T19:09:50.224692+0000","last_undegraded":"2024-03-22T19:09:50.224692+0000","last_fullsized":"2024-03-22T19:09:50.224692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:13:20.030654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230105+0000","last_change":"2024-03-22T19:09:13.652102+0000","last_active":"2024-03-22T19:09:50.230105+0000","last_peered":"2024-03-22T19:09:50.230105+0000","last_clean":"2024-03-22T19:09:50.230105+0000","last_became_active":"2024-03-22T19:09:13.650104+0000","last_became_peered":"2024-03-22T19:09:13.650104+0000","last_unstale":"2024-03-22T19:09:50.230105+0000","last_undegraded":"2024-03-22T19:09:50.230105+0000","last_fullsized":"2024-03-22T19:09:50.230105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:52:19.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228937+0000","last_change":"2024-03-22T19:09:13.639587+0000","last_active":"2024-03-22T19:09:50.228937+0000","last_peered":"2024-03-22T19:09:50.228937+0000","last_clean":"2024-03-22T19:09:50.228937+0000","last_became_active":"2024-03-22T19:09:13.639278+0000","last_became_peered":"2024-03-22T19:09:13.639278+0000","last_unstale":"2024-03-22T19:09:50.228937+0000","last_undegraded":"2024-03-22T19:09:50.228937+0000","last_fullsized":"2024-03-22T19:09:50.228937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:43:27.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280115+0000","last_change":"2024-03-22T19:09:51.280115+0000","last_active":"2024-03-22T19:09:51.280115+0000","last_peered":"2024-03-22T19:09:51.280115+0000","last_clean":"2024-03-22T19:09:51.280115+0000","last_became_active":"2024-03-22T19:09:51.279764+0000","last_became_peered":"2024-03-22T19:09:51.279764+0000","last_unstale":"2024-03-22T19:09:51.280115+0000","last_undegraded":"2024-03-22T19:09:51.280115+0000","last_fullsized":"2024-03-22T19:09:51.280115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:51:35.601327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224581+0000","last_change":"2024-03-22T19:09:13.640960+0000","last_active":"2024-03-22T19:09:50.224581+0000","last_peered":"2024-03-22T19:09:50.224581+0000","last_clean":"2024-03-22T19:09:50.224581+0000","last_became_active":"2024-03-22T19:09:13.640472+0000","last_became_peered":"2024-03-22T19:09:13.640472+0000","last_unstale":"2024-03-22T19:09:50.224581+0000","last_undegraded":"2024-03-22T19:09:50.224581+0000","last_fullsized":"2024-03-22T19:09:50.224581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:43:20.336596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223728+0000","last_change":"2024-03-22T19:09:13.648546+0000","last_active":"2024-03-22T19:09:50.223728+0000","last_peered":"2024-03-22T19:09:50.223728+0000","last_clean":"2024-03-22T19:09:50.223728+0000","last_became_active":"2024-03-22T19:09:13.648026+0000","last_became_peered":"2024-03-22T19:09:13.648026+0000","last_unstale":"2024-03-22T19:09:50.223728+0000","last_undegraded":"2024-03-22T19:09:50.223728+0000","last_fullsized":"2024-03-22T19:09:50.223728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:28:52.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221704+0000","last_change":"2024-03-22T19:09:13.646911+0000","last_active":"2024-03-22T19:09:50.221704+0000","last_peered":"2024-03-22T19:09:50.221704+0000","last_clean":"2024-03-22T19:09:50.221704+0000","last_became_active":"2024-03-22T19:09:13.643562+0000","last_became_peered":"2024-03-22T19:09:13.643562+0000","last_unstale":"2024-03-22T19:09:50.221704+0000","last_undegraded":"2024-03-22T19:09:50.221704+0000","last_fullsized":"2024-03-22T19:09:50.221704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:07:33.271199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224532+0000","last_change":"2024-03-22T19:09:13.639316+0000","last_active":"2024-03-22T19:09:50.224532+0000","last_peered":"2024-03-22T19:09:50.224532+0000","last_clean":"2024-03-22T19:09:50.224532+0000","last_became_active":"2024-03-22T19:09:13.637480+0000","last_became_peered":"2024-03-22T19:09:13.637480+0000","last_unstale":"2024-03-22T19:09:50.224532+0000","last_undegraded":"2024-03-22T19:09:50.224532+0000","last_fullsized":"2024-03-22T19:09:50.224532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:46:45.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225345+0000","last_change":"2024-03-22T19:09:13.641932+0000","last_active":"2024-03-22T19:09:50.225345+0000","last_peered":"2024-03-22T19:09:50.225345+0000","last_clean":"2024-03-22T19:09:50.225345+0000","last_became_active":"2024-03-22T19:09:13.641605+0000","last_became_peered":"2024-03-22T19:09:13.641605+0000","last_unstale":"2024-03-22T19:09:50.225345+0000","last_undegraded":"2024-03-22T19:09:50.225345+0000","last_fullsized":"2024-03-22T19:09:50.225345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:42:34.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228414+0000","last_change":"2024-03-22T19:09:13.644509+0000","last_active":"2024-03-22T19:09:50.228414+0000","last_peered":"2024-03-22T19:09:50.228414+0000","last_clean":"2024-03-22T19:09:50.228414+0000","last_became_active":"2024-03-22T19:09:13.644171+0000","last_became_peered":"2024-03-22T19:09:13.644171+0000","last_unstale":"2024-03-22T19:09:50.228414+0000","last_undegraded":"2024-03-22T19:09:50.228414+0000","last_fullsized":"2024-03-22T19:09:50.228414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:19:49.195436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229938+0000","last_change":"2024-03-22T19:09:13.648840+0000","last_active":"2024-03-22T19:09:50.229938+0000","last_peered":"2024-03-22T19:09:50.229938+0000","last_clean":"2024-03-22T19:09:50.229938+0000","last_became_active":"2024-03-22T19:09:13.648463+0000","last_became_peered":"2024-03-22T19:09:13.648463+0000","last_unstale":"2024-03-22T19:09:50.229938+0000","last_undegraded":"2024-03-22T19:09:50.229938+0000","last_fullsized":"2024-03-22T19:09:50.229938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:23:00.857778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224855+0000","last_change":"2024-03-22T19:09:13.640227+0000","last_active":"2024-03-22T19:09:50.224855+0000","last_peered":"2024-03-22T19:09:50.224855+0000","last_clean":"2024-03-22T19:09:50.224855+0000","last_became_active":"2024-03-22T19:09:13.639902+0000","last_became_peered":"2024-03-22T19:09:13.639902+0000","last_unstale":"2024-03-22T19:09:50.224855+0000","last_undegraded":"2024-03-22T19:09:50.224855+0000","last_fullsized":"2024-03-22T19:09:50.224855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:30.473229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226114+0000","last_change":"2024-03-22T19:09:13.653545+0000","last_active":"2024-03-22T19:09:50.226114+0000","last_peered":"2024-03-22T19:09:50.226114+0000","last_clean":"2024-03-22T19:09:50.226114+0000","last_became_active":"2024-03-22T19:09:13.653127+0000","last_became_peered":"2024-03-22T19:09:13.653127+0000","last_unstale":"2024-03-22T19:09:50.226114+0000","last_undegraded":"2024-03-22T19:09:50.226114+0000","last_fullsized":"2024-03-22T19:09:50.226114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:33.703638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:44.232970+0000","last_change":"2024-03-22T19:09:13.136771+0000","last_active":"2024-03-22T19:10:44.232970+0000","last_peered":"2024-03-22T19:10:44.232970+0000","last_clean":"2024-03-22T19:10:44.232970+0000","last_became_active":"2024-03-22T19:09:13.136054+0000","last_became_peered":"2024-03-22T19:09:13.136054+0000","last_unstale":"2024-03-22T19:10:44.232970+0000","last_undegraded":"2024-03-22T19:10:44.232970+0000","last_fullsized":"2024-03-22T19:10:44.232970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:11:52.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230751+0000","last_change":"2024-03-22T19:09:13.658342+0000","last_active":"2024-03-22T19:09:50.230751+0000","last_peered":"2024-03-22T19:09:50.230751+0000","last_clean":"2024-03-22T19:09:50.230751+0000","last_became_active":"2024-03-22T19:09:13.658046+0000","last_became_peered":"2024-03-22T19:09:13.658046+0000","last_unstale":"2024-03-22T19:09:50.230751+0000","last_undegraded":"2024-03-22T19:09:50.230751+0000","last_fullsized":"2024-03-22T19:09:50.230751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:42:43.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230850+0000","last_change":"2024-03-22T19:09:13.150070+0000","last_active":"2024-03-22T19:09:50.230850+0000","last_peered":"2024-03-22T19:09:50.230850+0000","last_clean":"2024-03-22T19:09:50.230850+0000","last_became_active":"2024-03-22T19:09:13.149296+0000","last_became_peered":"2024-03-22T19:09:13.149296+0000","last_unstale":"2024-03-22T19:09:50.230850+0000","last_undegraded":"2024-03-22T19:09:50.230850+0000","last_fullsized":"2024-03-22T19:09:50.230850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:21:45.232734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226153+0000","last_change":"2024-03-22T19:09:13.649174+0000","last_active":"2024-03-22T19:09:50.226153+0000","last_peered":"2024-03-22T19:09:50.226153+0000","last_clean":"2024-03-22T19:09:50.226153+0000","last_became_active":"2024-03-22T19:09:13.648876+0000","last_became_peered":"2024-03-22T19:09:13.648876+0000","last_unstale":"2024-03-22T19:09:50.226153+0000","last_undegraded":"2024-03-22T19:09:50.226153+0000","last_fullsized":"2024-03-22T19:09:50.226153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:46:39.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231344+0000","last_change":"2024-03-22T19:09:13.142424+0000","last_active":"2024-03-22T19:09:50.231344+0000","last_peered":"2024-03-22T19:09:50.231344+0000","last_clean":"2024-03-22T19:09:50.231344+0000","last_became_active":"2024-03-22T19:09:13.142066+0000","last_became_peered":"2024-03-22T19:09:13.142066+0000","last_unstale":"2024-03-22T19:09:50.231344+0000","last_undegraded":"2024-03-22T19:09:50.231344+0000","last_fullsized":"2024-03-22T19:09:50.231344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:55:00.545905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228584+0000","last_change":"2024-03-22T19:09:13.641253+0000","last_active":"2024-03-22T19:09:50.228584+0000","last_peered":"2024-03-22T19:09:50.228584+0000","last_clean":"2024-03-22T19:09:50.228584+0000","last_became_active":"2024-03-22T19:09:13.640947+0000","last_became_peered":"2024-03-22T19:09:13.640947+0000","last_unstale":"2024-03-22T19:09:50.228584+0000","last_undegraded":"2024-03-22T19:09:50.228584+0000","last_fullsized":"2024-03-22T19:09:50.228584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:40:52.223100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231109+0000","last_change":"2024-03-22T19:09:13.136861+0000","last_active":"2024-03-22T19:09:50.231109+0000","last_peered":"2024-03-22T19:09:50.231109+0000","last_clean":"2024-03-22T19:09:50.231109+0000","last_became_active":"2024-03-22T19:09:13.136514+0000","last_became_peered":"2024-03-22T19:09:13.136514+0000","last_unstale":"2024-03-22T19:09:50.231109+0000","last_undegraded":"2024-03-22T19:09:50.231109+0000","last_fullsized":"2024-03-22T19:09:50.231109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:01:28.290840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232324+0000","last_change":"2024-03-22T19:09:13.650277+0000","last_active":"2024-03-22T19:09:50.232324+0000","last_peered":"2024-03-22T19:09:50.232324+0000","last_clean":"2024-03-22T19:09:50.232324+0000","last_became_active":"2024-03-22T19:09:13.649974+0000","last_became_peered":"2024-03-22T19:09:13.649974+0000","last_unstale":"2024-03-22T19:09:50.232324+0000","last_undegraded":"2024-03-22T19:09:50.232324+0000","last_fullsized":"2024-03-22T19:09:50.232324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:30:00.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:45.234575+0000","last_change":"2024-03-22T19:09:13.140115+0000","last_active":"2024-03-22T19:10:45.234575+0000","last_peered":"2024-03-22T19:10:45.234575+0000","last_clean":"2024-03-22T19:10:45.234575+0000","last_became_active":"2024-03-22T19:09:13.139740+0000","last_became_peered":"2024-03-22T19:09:13.139740+0000","last_unstale":"2024-03-22T19:10:45.234575+0000","last_undegraded":"2024-03-22T19:10:45.234575+0000","last_fullsized":"2024-03-22T19:10:45.234575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:33:06.275148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232010+0000","last_change":"2024-03-22T19:09:13.646300+0000","last_active":"2024-03-22T19:09:50.232010+0000","last_peered":"2024-03-22T19:09:50.232010+0000","last_clean":"2024-03-22T19:09:50.232010+0000","last_became_active":"2024-03-22T19:09:13.645220+0000","last_became_peered":"2024-03-22T19:09:13.645220+0000","last_unstale":"2024-03-22T19:09:50.232010+0000","last_undegraded":"2024-03-22T19:09:50.232010+0000","last_fullsized":"2024-03-22T19:09:50.232010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:10:52.085158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227866+0000","last_change":"2024-03-22T19:09:13.140887+0000","last_active":"2024-03-22T19:09:50.227866+0000","last_peered":"2024-03-22T19:09:50.227866+0000","last_clean":"2024-03-22T19:09:50.227866+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:09:50.227866+0000","last_undegraded":"2024-03-22T19:09:50.227866+0000","last_fullsized":"2024-03-22T19:09:50.227866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:07:10.810819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230805+0000","last_change":"2024-03-22T19:09:13.647429+0000","last_active":"2024-03-22T19:09:50.230805+0000","last_peered":"2024-03-22T19:09:50.230805+0000","last_clean":"2024-03-22T19:09:50.230805+0000","last_became_active":"2024-03-22T19:09:13.647120+0000","last_became_peered":"2024-03-22T19:09:13.647120+0000","last_unstale":"2024-03-22T19:09:50.230805+0000","last_undegraded":"2024-03-22T19:09:50.230805+0000","last_fullsized":"2024-03-22T19:09:50.230805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:55:17.050519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221505+0000","last_change":"2024-03-22T19:09:13.133959+0000","last_active":"2024-03-22T19:09:50.221505+0000","last_peered":"2024-03-22T19:09:50.221505+0000","last_clean":"2024-03-22T19:09:50.221505+0000","last_became_active":"2024-03-22T19:09:13.133612+0000","last_became_peered":"2024-03-22T19:09:13.133612+0000","last_unstale":"2024-03-22T19:09:50.221505+0000","last_undegraded":"2024-03-22T19:09:50.221505+0000","last_fullsized":"2024-03-22T19:09:50.221505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:56:10.807929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230067+0000","last_change":"2024-03-22T19:09:13.641858+0000","last_active":"2024-03-22T19:09:50.230067+0000","last_peered":"2024-03-22T19:09:50.230067+0000","last_clean":"2024-03-22T19:09:50.230067+0000","last_became_active":"2024-03-22T19:09:13.641561+0000","last_became_peered":"2024-03-22T19:09:13.641561+0000","last_unstale":"2024-03-22T19:09:50.230067+0000","last_undegraded":"2024-03-22T19:09:50.230067+0000","last_fullsized":"2024-03-22T19:09:50.230067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:38:14.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229198+0000","last_change":"2024-03-22T19:09:13.139570+0000","last_active":"2024-03-22T19:09:50.229198+0000","last_peered":"2024-03-22T19:09:50.229198+0000","last_clean":"2024-03-22T19:09:50.229198+0000","last_became_active":"2024-03-22T19:09:13.139194+0000","last_became_peered":"2024-03-22T19:09:13.139194+0000","last_unstale":"2024-03-22T19:09:50.229198+0000","last_undegraded":"2024-03-22T19:09:50.229198+0000","last_fullsized":"2024-03-22T19:09:50.229198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:02:14.166746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224276+0000","last_change":"2024-03-22T19:09:13.641805+0000","last_active":"2024-03-22T19:09:50.224276+0000","last_peered":"2024-03-22T19:09:50.224276+0000","last_clean":"2024-03-22T19:09:50.224276+0000","last_became_active":"2024-03-22T19:09:13.641469+0000","last_became_peered":"2024-03-22T19:09:13.641469+0000","last_unstale":"2024-03-22T19:09:50.224276+0000","last_undegraded":"2024-03-22T19:09:50.224276+0000","last_fullsized":"2024-03-22T19:09:50.224276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:28.599541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":190,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:46.235367+0000","last_change":"2024-03-22T19:09:13.140562+0000","last_active":"2024-03-22T19:10:46.235367+0000","last_peered":"2024-03-22T19:10:46.235367+0000","last_clean":"2024-03-22T19:10:46.235367+0000","last_became_active":"2024-03-22T19:09:13.140235+0000","last_became_peered":"2024-03-22T19:09:13.140235+0000","last_unstale":"2024-03-22T19:10:46.235367+0000","last_undegraded":"2024-03-22T19:10:46.235367+0000","last_fullsized":"2024-03-22T19:10:46.235367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:35:51.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228903+0000","last_change":"2024-03-22T19:09:13.642900+0000","last_active":"2024-03-22T19:09:50.228903+0000","last_peered":"2024-03-22T19:09:50.228903+0000","last_clean":"2024-03-22T19:09:50.228903+0000","last_became_active":"2024-03-22T19:09:13.642575+0000","last_became_peered":"2024-03-22T19:09:13.642575+0000","last_unstale":"2024-03-22T19:09:50.228903+0000","last_undegraded":"2024-03-22T19:09:50.228903+0000","last_fullsized":"2024-03-22T19:09:50.228903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:51:04.658690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.438753+0000","last_change":"2024-03-22T19:09:51.438753+0000","last_active":"2024-03-22T19:09:51.438753+0000","last_peered":"2024-03-22T19:09:51.438753+0000","last_clean":"2024-03-22T19:09:51.438753+0000","last_became_active":"2024-03-22T19:09:51.413301+0000","last_became_peered":"2024-03-22T19:09:51.413301+0000","last_unstale":"2024-03-22T19:09:51.438753+0000","last_undegraded":"2024-03-22T19:09:51.438753+0000","last_fullsized":"2024-03-22T19:09:51.438753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:49:49.994282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226106+0000","last_change":"2024-03-22T19:09:13.651342+0000","last_active":"2024-03-22T19:09:50.226106+0000","last_peered":"2024-03-22T19:09:50.226106+0000","last_clean":"2024-03-22T19:09:50.226106+0000","last_became_active":"2024-03-22T19:09:13.650847+0000","last_became_peered":"2024-03-22T19:09:13.650847+0000","last_unstale":"2024-03-22T19:09:50.226106+0000","last_undegraded":"2024-03-22T19:09:50.226106+0000","last_fullsized":"2024-03-22T19:09:50.226106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:38.243760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232420+0000","last_change":"2024-03-22T19:09:13.145962+0000","last_active":"2024-03-22T19:09:50.232420+0000","last_peered":"2024-03-22T19:09:50.232420+0000","last_clean":"2024-03-22T19:09:50.232420+0000","last_became_active":"2024-03-22T19:09:13.145568+0000","last_became_peered":"2024-03-22T19:09:13.145568+0000","last_unstale":"2024-03-22T19:09:50.232420+0000","last_undegraded":"2024-03-22T19:09:50.232420+0000","last_fullsized":"2024-03-22T19:09:50.232420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:59:47.266721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228700+0000","last_change":"2024-03-22T19:09:13.647447+0000","last_active":"2024-03-22T19:09:50.228700+0000","last_peered":"2024-03-22T19:09:50.228700+0000","last_clean":"2024-03-22T19:09:50.228700+0000","last_became_active":"2024-03-22T19:09:13.647114+0000","last_became_peered":"2024-03-22T19:09:13.647114+0000","last_unstale":"2024-03-22T19:09:50.228700+0000","last_undegraded":"2024-03-22T19:09:50.228700+0000","last_fullsized":"2024-03-22T19:09:50.228700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:31:06.978230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":223,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:46.250848+0000","last_change":"2024-03-22T19:09:13.142291+0000","last_active":"2024-03-22T19:10:46.250848+0000","last_peered":"2024-03-22T19:10:46.250848+0000","last_clean":"2024-03-22T19:10:46.250848+0000","last_became_active":"2024-03-22T19:09:13.141887+0000","last_became_peered":"2024-03-22T19:09:13.141887+0000","last_unstale":"2024-03-22T19:10:46.250848+0000","last_undegraded":"2024-03-22T19:10:46.250848+0000","last_fullsized":"2024-03-22T19:10:46.250848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:26.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59653,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":148,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230888+0000","last_change":"2024-03-22T19:09:13.638864+0000","last_active":"2024-03-22T19:09:50.230888+0000","last_peered":"2024-03-22T19:09:50.230888+0000","last_clean":"2024-03-22T19:09:50.230888+0000","last_became_active":"2024-03-22T19:09:13.638515+0000","last_became_peered":"2024-03-22T19:09:13.638515+0000","last_unstale":"2024-03-22T19:09:50.230888+0000","last_undegraded":"2024-03-22T19:09:50.230888+0000","last_fullsized":"2024-03-22T19:09:50.230888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:09.946192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226121+0000","last_change":"2024-03-22T19:09:13.144423+0000","last_active":"2024-03-22T19:09:50.226121+0000","last_peered":"2024-03-22T19:09:50.226121+0000","last_clean":"2024-03-22T19:09:50.226121+0000","last_became_active":"2024-03-22T19:09:13.144066+0000","last_became_peered":"2024-03-22T19:09:13.144066+0000","last_unstale":"2024-03-22T19:09:50.226121+0000","last_undegraded":"2024-03-22T19:09:50.226121+0000","last_fullsized":"2024-03-22T19:09:50.226121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:31:11.820490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221747+0000","last_change":"2024-03-22T19:09:13.642945+0000","last_active":"2024-03-22T19:09:50.221747+0000","last_peered":"2024-03-22T19:09:50.221747+0000","last_clean":"2024-03-22T19:09:50.221747+0000","last_became_active":"2024-03-22T19:09:13.641545+0000","last_became_peered":"2024-03-22T19:09:13.641545+0000","last_unstale":"2024-03-22T19:09:50.221747+0000","last_undegraded":"2024-03-22T19:09:50.221747+0000","last_fullsized":"2024-03-22T19:09:50.221747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:14:23.667310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230590+0000","last_change":"2024-03-22T19:09:13.148547+0000","last_active":"2024-03-22T19:09:50.230590+0000","last_peered":"2024-03-22T19:09:50.230590+0000","last_clean":"2024-03-22T19:09:50.230590+0000","last_became_active":"2024-03-22T19:09:13.148274+0000","last_became_peered":"2024-03-22T19:09:13.148274+0000","last_unstale":"2024-03-22T19:09:50.230590+0000","last_undegraded":"2024-03-22T19:09:50.230590+0000","last_fullsized":"2024-03-22T19:09:50.230590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:58.961482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224672+0000","last_change":"2024-03-22T19:09:13.649426+0000","last_active":"2024-03-22T19:09:50.224672+0000","last_peered":"2024-03-22T19:09:50.224672+0000","last_clean":"2024-03-22T19:09:50.224672+0000","last_became_active":"2024-03-22T19:09:13.648988+0000","last_became_peered":"2024-03-22T19:09:13.648988+0000","last_unstale":"2024-03-22T19:09:50.224672+0000","last_undegraded":"2024-03-22T19:09:50.224672+0000","last_fullsized":"2024-03-22T19:09:50.224672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:49:54.075552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:43.232065+0000","last_change":"2024-03-22T19:09:13.138725+0000","last_active":"2024-03-22T19:10:43.232065+0000","last_peered":"2024-03-22T19:10:43.232065+0000","last_clean":"2024-03-22T19:10:43.232065+0000","last_became_active":"2024-03-22T19:09:13.138434+0000","last_became_peered":"2024-03-22T19:09:13.138434+0000","last_unstale":"2024-03-22T19:10:43.232065+0000","last_undegraded":"2024-03-22T19:10:43.232065+0000","last_fullsized":"2024-03-22T19:10:43.232065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:39:09.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.638620+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.638312+0000","last_became_peered":"2024-03-22T19:09:13.638312+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:33:16.404499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.435708+0000","last_change":"2024-03-22T19:09:51.435708+0000","last_active":"2024-03-22T19:09:51.435708+0000","last_peered":"2024-03-22T19:09:51.435708+0000","last_clean":"2024-03-22T19:09:51.435708+0000","last_became_active":"2024-03-22T19:09:51.412550+0000","last_became_peered":"2024-03-22T19:09:51.412550+0000","last_unstale":"2024-03-22T19:09:51.435708+0000","last_undegraded":"2024-03-22T19:09:51.435708+0000","last_fullsized":"2024-03-22T19:09:51.435708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:11:44.092847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229832+0000","last_change":"2024-03-22T19:09:13.644207+0000","last_active":"2024-03-22T19:09:50.229832+0000","last_peered":"2024-03-22T19:09:50.229832+0000","last_clean":"2024-03-22T19:09:50.229832+0000","last_became_active":"2024-03-22T19:09:13.643766+0000","last_became_peered":"2024-03-22T19:09:13.643766+0000","last_unstale":"2024-03-22T19:09:50.229832+0000","last_undegraded":"2024-03-22T19:09:50.229832+0000","last_fullsized":"2024-03-22T19:09:50.229832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:50:42.579148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231493+0000","last_change":"2024-03-22T19:09:13.138737+0000","last_active":"2024-03-22T19:09:50.231493+0000","last_peered":"2024-03-22T19:09:50.231493+0000","last_clean":"2024-03-22T19:09:50.231493+0000","last_became_active":"2024-03-22T19:09:13.138402+0000","last_became_peered":"2024-03-22T19:09:13.138402+0000","last_unstale":"2024-03-22T19:09:50.231493+0000","last_undegraded":"2024-03-22T19:09:50.231493+0000","last_fullsized":"2024-03-22T19:09:50.231493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:11:41.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229589+0000","last_change":"2024-03-22T19:09:13.646477+0000","last_active":"2024-03-22T19:09:50.229589+0000","last_peered":"2024-03-22T19:09:50.229589+0000","last_clean":"2024-03-22T19:09:50.229589+0000","last_became_active":"2024-03-22T19:09:13.645980+0000","last_became_peered":"2024-03-22T19:09:13.645980+0000","last_unstale":"2024-03-22T19:09:50.229589+0000","last_undegraded":"2024-03-22T19:09:50.229589+0000","last_fullsized":"2024-03-22T19:09:50.229589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:20.569807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225716+0000","last_change":"2024-03-22T19:09:13.145384+0000","last_active":"2024-03-22T19:09:50.225716+0000","last_peered":"2024-03-22T19:09:50.225716+0000","last_clean":"2024-03-22T19:09:50.225716+0000","last_became_active":"2024-03-22T19:09:13.144955+0000","last_became_peered":"2024-03-22T19:09:13.144955+0000","last_unstale":"2024-03-22T19:09:50.225716+0000","last_undegraded":"2024-03-22T19:09:50.225716+0000","last_fullsized":"2024-03-22T19:09:50.225716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:08:11.947354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227355+0000","last_change":"2024-03-22T19:09:13.647374+0000","last_active":"2024-03-22T19:09:50.227355+0000","last_peered":"2024-03-22T19:09:50.227355+0000","last_clean":"2024-03-22T19:09:50.227355+0000","last_became_active":"2024-03-22T19:09:13.647031+0000","last_became_peered":"2024-03-22T19:09:13.647031+0000","last_unstale":"2024-03-22T19:09:50.227355+0000","last_undegraded":"2024-03-22T19:09:50.227355+0000","last_fullsized":"2024-03-22T19:09:50.227355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:22:28.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:45.234724+0000","last_change":"2024-03-22T19:09:13.135904+0000","last_active":"2024-03-22T19:10:45.234724+0000","last_peered":"2024-03-22T19:10:45.234724+0000","last_clean":"2024-03-22T19:10:45.234724+0000","last_became_active":"2024-03-22T19:09:13.135584+0000","last_became_peered":"2024-03-22T19:09:13.135584+0000","last_unstale":"2024-03-22T19:10:45.234724+0000","last_undegraded":"2024-03-22T19:10:45.234724+0000","last_fullsized":"2024-03-22T19:10:45.234724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:57:57.138406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.227464+0000","last_change":"2024-03-22T19:09:13.647011+0000","last_active":"2024-03-22T19:09:50.227464+0000","last_peered":"2024-03-22T19:09:50.227464+0000","last_clean":"2024-03-22T19:09:50.227464+0000","last_became_active":"2024-03-22T19:09:13.646643+0000","last_became_peered":"2024-03-22T19:09:13.646643+0000","last_unstale":"2024-03-22T19:09:50.227464+0000","last_undegraded":"2024-03-22T19:09:50.227464+0000","last_fullsized":"2024-03-22T19:09:50.227464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:12:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229382+0000","last_change":"2024-03-22T19:09:13.644373+0000","last_active":"2024-03-22T19:09:50.229382+0000","last_peered":"2024-03-22T19:09:50.229382+0000","last_clean":"2024-03-22T19:09:50.229382+0000","last_became_active":"2024-03-22T19:09:13.642469+0000","last_became_peered":"2024-03-22T19:09:13.642469+0000","last_unstale":"2024-03-22T19:09:50.229382+0000","last_undegraded":"2024-03-22T19:09:50.229382+0000","last_fullsized":"2024-03-22T19:09:50.229382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:25:13.498420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:43.231386+0000","last_change":"2024-03-22T19:09:13.147482+0000","last_active":"2024-03-22T19:10:43.231386+0000","last_peered":"2024-03-22T19:10:43.231386+0000","last_clean":"2024-03-22T19:10:43.231386+0000","last_became_active":"2024-03-22T19:09:13.147115+0000","last_became_peered":"2024-03-22T19:09:13.147115+0000","last_unstale":"2024-03-22T19:10:43.231386+0000","last_undegraded":"2024-03-22T19:10:43.231386+0000","last_fullsized":"2024-03-22T19:10:43.231386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:20:15.722248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230745+0000","last_change":"2024-03-22T19:09:13.655051+0000","last_active":"2024-03-22T19:09:50.230745+0000","last_peered":"2024-03-22T19:09:50.230745+0000","last_clean":"2024-03-22T19:09:50.230745+0000","last_became_active":"2024-03-22T19:09:13.654714+0000","last_became_peered":"2024-03-22T19:09:13.654714+0000","last_unstale":"2024-03-22T19:09:50.230745+0000","last_undegraded":"2024-03-22T19:09:50.230745+0000","last_fullsized":"2024-03-22T19:09:50.230745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:48:45.033398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223407+0000","last_change":"2024-03-22T19:09:13.136526+0000","last_active":"2024-03-22T19:09:50.223407+0000","last_peered":"2024-03-22T19:09:50.223407+0000","last_clean":"2024-03-22T19:09:50.223407+0000","last_became_active":"2024-03-22T19:09:13.136166+0000","last_became_peered":"2024-03-22T19:09:13.136166+0000","last_unstale":"2024-03-22T19:09:50.223407+0000","last_undegraded":"2024-03-22T19:09:50.223407+0000","last_fullsized":"2024-03-22T19:09:50.223407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:53:17.196729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224404+0000","last_change":"2024-03-22T19:08:20.774722+0000","last_active":"2024-03-22T19:09:50.224404+0000","last_peered":"2024-03-22T19:09:50.224404+0000","last_clean":"2024-03-22T19:09:50.224404+0000","last_became_active":"2024-03-22T19:08:20.774258+0000","last_became_peered":"2024-03-22T19:08:20.774258+0000","last_unstale":"2024-03-22T19:09:50.224404+0000","last_undegraded":"2024-03-22T19:09:50.224404+0000","last_fullsized":"2024-03-22T19:09:50.224404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:05:46.700251+0000","last_clean_scrub_stamp":"2024-03-22T19:05:46.700251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:38:44.231826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221409+0000","last_change":"2024-03-22T19:09:13.638705+0000","last_active":"2024-03-22T19:09:50.221409+0000","last_peered":"2024-03-22T19:09:50.221409+0000","last_clean":"2024-03-22T19:09:50.221409+0000","last_became_active":"2024-03-22T19:09:13.638371+0000","last_became_peered":"2024-03-22T19:09:13.638371+0000","last_unstale":"2024-03-22T19:09:50.221409+0000","last_undegraded":"2024-03-22T19:09:50.221409+0000","last_fullsized":"2024-03-22T19:09:50.221409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:32:24.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224573+0000","last_change":"2024-03-22T19:09:13.137536+0000","last_active":"2024-03-22T19:09:50.224573+0000","last_peered":"2024-03-22T19:09:50.224573+0000","last_clean":"2024-03-22T19:09:50.224573+0000","last_became_active":"2024-03-22T19:09:13.137118+0000","last_became_peered":"2024-03-22T19:09:13.137118+0000","last_unstale":"2024-03-22T19:09:50.224573+0000","last_undegraded":"2024-03-22T19:09:50.224573+0000","last_fullsized":"2024-03-22T19:09:50.224573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:44:34.719211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231242+0000","last_change":"2024-03-22T19:09:13.147588+0000","last_active":"2024-03-22T19:09:50.231242+0000","last_peered":"2024-03-22T19:09:50.231242+0000","last_clean":"2024-03-22T19:09:50.231242+0000","last_became_active":"2024-03-22T19:09:13.147277+0000","last_became_peered":"2024-03-22T19:09:13.147277+0000","last_unstale":"2024-03-22T19:09:50.231242+0000","last_undegraded":"2024-03-22T19:09:50.231242+0000","last_fullsized":"2024-03-22T19:09:50.231242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:49:11.995556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221846+0000","last_change":"2024-03-22T19:09:13.638639+0000","last_active":"2024-03-22T19:09:50.221846+0000","last_peered":"2024-03-22T19:09:50.221846+0000","last_clean":"2024-03-22T19:09:50.221846+0000","last_became_active":"2024-03-22T19:09:13.638306+0000","last_became_peered":"2024-03-22T19:09:13.638306+0000","last_unstale":"2024-03-22T19:09:50.221846+0000","last_undegraded":"2024-03-22T19:09:50.221846+0000","last_fullsized":"2024-03-22T19:09:50.221846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:11:10.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224775+0000","last_change":"2024-03-22T19:09:13.151023+0000","last_active":"2024-03-22T19:09:50.224775+0000","last_peered":"2024-03-22T19:09:50.224775+0000","last_clean":"2024-03-22T19:09:50.224775+0000","last_became_active":"2024-03-22T19:09:13.150561+0000","last_became_peered":"2024-03-22T19:09:13.150561+0000","last_unstale":"2024-03-22T19:09:50.224775+0000","last_undegraded":"2024-03-22T19:09:50.224775+0000","last_fullsized":"2024-03-22T19:09:50.224775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:18:17.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230722+0000","last_change":"2024-03-22T19:09:13.658668+0000","last_active":"2024-03-22T19:09:50.230722+0000","last_peered":"2024-03-22T19:09:50.230722+0000","last_clean":"2024-03-22T19:09:50.230722+0000","last_became_active":"2024-03-22T19:09:13.656608+0000","last_became_peered":"2024-03-22T19:09:13.656608+0000","last_unstale":"2024-03-22T19:09:50.230722+0000","last_undegraded":"2024-03-22T19:09:50.230722+0000","last_fullsized":"2024-03-22T19:09:50.230722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:57.836048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.223816+0000","last_change":"2024-03-22T19:09:13.137581+0000","last_active":"2024-03-22T19:09:50.223816+0000","last_peered":"2024-03-22T19:09:50.223816+0000","last_clean":"2024-03-22T19:09:50.223816+0000","last_became_active":"2024-03-22T19:09:13.137232+0000","last_became_peered":"2024-03-22T19:09:13.137232+0000","last_unstale":"2024-03-22T19:09:50.223816+0000","last_undegraded":"2024-03-22T19:09:50.223816+0000","last_fullsized":"2024-03-22T19:09:50.223816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:16:41.091991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225947+0000","last_change":"2024-03-22T19:09:13.651589+0000","last_active":"2024-03-22T19:09:50.225947+0000","last_peered":"2024-03-22T19:09:50.225947+0000","last_clean":"2024-03-22T19:09:50.225947+0000","last_became_active":"2024-03-22T19:09:13.651225+0000","last_became_peered":"2024-03-22T19:09:13.651225+0000","last_unstale":"2024-03-22T19:09:50.225947+0000","last_undegraded":"2024-03-22T19:09:50.225947+0000","last_fullsized":"2024-03-22T19:09:50.225947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:05:15.329219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228471+0000","last_change":"2024-03-22T19:09:13.140305+0000","last_active":"2024-03-22T19:09:50.228471+0000","last_peered":"2024-03-22T19:09:50.228471+0000","last_clean":"2024-03-22T19:09:50.228471+0000","last_became_active":"2024-03-22T19:09:13.140010+0000","last_became_peered":"2024-03-22T19:09:13.140010+0000","last_unstale":"2024-03-22T19:09:50.228471+0000","last_undegraded":"2024-03-22T19:09:50.228471+0000","last_fullsized":"2024-03-22T19:09:50.228471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:09:00.125285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230778+0000","last_change":"2024-03-22T19:09:13.637171+0000","last_active":"2024-03-22T19:09:50.230778+0000","last_peered":"2024-03-22T19:09:50.230778+0000","last_clean":"2024-03-22T19:09:50.230778+0000","last_became_active":"2024-03-22T19:09:13.636822+0000","last_became_peered":"2024-03-22T19:09:13.636822+0000","last_unstale":"2024-03-22T19:09:50.230778+0000","last_undegraded":"2024-03-22T19:09:50.230778+0000","last_fullsized":"2024-03-22T19:09:50.230778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:59:46.476662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230201+0000","last_change":"2024-03-22T19:09:13.149177+0000","last_active":"2024-03-22T19:09:50.230201+0000","last_peered":"2024-03-22T19:09:50.230201+0000","last_clean":"2024-03-22T19:09:50.230201+0000","last_became_active":"2024-03-22T19:09:13.148626+0000","last_became_peered":"2024-03-22T19:09:13.148626+0000","last_unstale":"2024-03-22T19:09:50.230201+0000","last_undegraded":"2024-03-22T19:09:50.230201+0000","last_fullsized":"2024-03-22T19:09:50.230201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:52:16.355256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229252+0000","last_change":"2024-03-22T19:09:13.652193+0000","last_active":"2024-03-22T19:09:50.229252+0000","last_peered":"2024-03-22T19:09:50.229252+0000","last_clean":"2024-03-22T19:09:50.229252+0000","last_became_active":"2024-03-22T19:09:13.651802+0000","last_became_peered":"2024-03-22T19:09:13.651802+0000","last_unstale":"2024-03-22T19:09:50.229252+0000","last_undegraded":"2024-03-22T19:09:50.229252+0000","last_fullsized":"2024-03-22T19:09:50.229252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:02.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.241586+0000","last_change":"2024-03-22T19:09:51.241586+0000","last_active":"2024-03-22T19:09:51.241586+0000","last_peered":"2024-03-22T19:09:51.241586+0000","last_clean":"2024-03-22T19:09:51.241586+0000","last_became_active":"2024-03-22T19:09:51.235327+0000","last_became_peered":"2024-03-22T19:09:51.235327+0000","last_unstale":"2024-03-22T19:09:51.241586+0000","last_undegraded":"2024-03-22T19:09:51.241586+0000","last_fullsized":"2024-03-22T19:09:51.241586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:43:59.307732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230257+0000","last_change":"2024-03-22T19:09:13.658645+0000","last_active":"2024-03-22T19:09:50.230257+0000","last_peered":"2024-03-22T19:09:50.230257+0000","last_clean":"2024-03-22T19:09:50.230257+0000","last_became_active":"2024-03-22T19:09:13.658283+0000","last_became_peered":"2024-03-22T19:09:13.658283+0000","last_unstale":"2024-03-22T19:09:50.230257+0000","last_undegraded":"2024-03-22T19:09:50.230257+0000","last_fullsized":"2024-03-22T19:09:50.230257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:43:03.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.231141+0000","last_change":"2024-03-22T19:09:13.143241+0000","last_active":"2024-03-22T19:09:50.231141+0000","last_peered":"2024-03-22T19:09:50.231141+0000","last_clean":"2024-03-22T19:09:50.231141+0000","last_became_active":"2024-03-22T19:09:13.142831+0000","last_became_peered":"2024-03-22T19:09:13.142831+0000","last_unstale":"2024-03-22T19:09:50.231141+0000","last_undegraded":"2024-03-22T19:09:50.231141+0000","last_fullsized":"2024-03-22T19:09:50.231141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:26:46.998713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224983+0000","last_change":"2024-03-22T19:09:13.650601+0000","last_active":"2024-03-22T19:09:50.224983+0000","last_peered":"2024-03-22T19:09:50.224983+0000","last_clean":"2024-03-22T19:09:50.224983+0000","last_became_active":"2024-03-22T19:09:13.649308+0000","last_became_peered":"2024-03-22T19:09:13.649308+0000","last_unstale":"2024-03-22T19:09:50.224983+0000","last_undegraded":"2024-03-22T19:09:50.224983+0000","last_fullsized":"2024-03-22T19:09:50.224983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T20:07:17.467531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":106,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:10:44.233437+0000","last_change":"2024-03-22T19:09:13.140988+0000","last_active":"2024-03-22T19:10:44.233437+0000","last_peered":"2024-03-22T19:10:44.233437+0000","last_clean":"2024-03-22T19:10:44.233437+0000","last_became_active":"2024-03-22T19:09:13.140578+0000","last_became_peered":"2024-03-22T19:09:13.140578+0000","last_unstale":"2024-03-22T19:10:44.233437+0000","last_undegraded":"2024-03-22T19:10:44.233437+0000","last_fullsized":"2024-03-22T19:10:44.233437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:23:36.113278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224702+0000","last_change":"2024-03-22T19:09:13.639777+0000","last_active":"2024-03-22T19:09:50.224702+0000","last_peered":"2024-03-22T19:09:50.224702+0000","last_clean":"2024-03-22T19:09:50.224702+0000","last_became_active":"2024-03-22T19:09:13.639415+0000","last_became_peered":"2024-03-22T19:09:13.639415+0000","last_unstale":"2024-03-22T19:09:50.224702+0000","last_undegraded":"2024-03-22T19:09:50.224702+0000","last_fullsized":"2024-03-22T19:09:50.224702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:59:52.456570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.232307+0000","last_change":"2024-03-22T19:09:13.150623+0000","last_active":"2024-03-22T19:09:50.232307+0000","last_peered":"2024-03-22T19:09:50.232307+0000","last_clean":"2024-03-22T19:09:50.232307+0000","last_became_active":"2024-03-22T19:09:13.150296+0000","last_became_peered":"2024-03-22T19:09:13.150296+0000","last_unstale":"2024-03-22T19:09:50.232307+0000","last_undegraded":"2024-03-22T19:09:50.232307+0000","last_fullsized":"2024-03-22T19:09:50.232307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:13:26.405345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224298+0000","last_change":"2024-03-22T19:09:13.650257+0000","last_active":"2024-03-22T19:09:50.224298+0000","last_peered":"2024-03-22T19:09:50.224298+0000","last_clean":"2024-03-22T19:09:50.224298+0000","last_became_active":"2024-03-22T19:09:13.649904+0000","last_became_peered":"2024-03-22T19:09:13.649904+0000","last_unstale":"2024-03-22T19:09:50.224298+0000","last_undegraded":"2024-03-22T19:09:50.224298+0000","last_fullsized":"2024-03-22T19:09:50.224298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T23:03:55.979210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221786+0000","last_change":"2024-03-22T19:09:13.142005+0000","last_active":"2024-03-22T19:09:50.221786+0000","last_peered":"2024-03-22T19:09:50.221786+0000","last_clean":"2024-03-22T19:09:50.221786+0000","last_became_active":"2024-03-22T19:09:13.141605+0000","last_became_peered":"2024-03-22T19:09:13.141605+0000","last_unstale":"2024-03-22T19:09:50.221786+0000","last_undegraded":"2024-03-22T19:09:50.221786+0000","last_fullsized":"2024-03-22T19:09:50.221786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:12.112793+0000","last_clean_scrub_stamp":"2024-03-22T19:09:12.112793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:02:28.331056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.224337+0000","last_change":"2024-03-22T19:09:13.636083+0000","last_active":"2024-03-22T19:09:50.224337+0000","last_peered":"2024-03-22T19:09:50.224337+0000","last_clean":"2024-03-22T19:09:50.224337+0000","last_became_active":"2024-03-22T19:09:13.635267+0000","last_became_peered":"2024-03-22T19:09:13.635267+0000","last_unstale":"2024-03-22T19:09:50.224337+0000","last_undegraded":"2024-03-22T19:09:50.224337+0000","last_fullsized":"2024-03-22T19:09:50.224337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:47:47.434902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229345+0000","last_change":"2024-03-22T19:09:13.653461+0000","last_active":"2024-03-22T19:09:50.229345+0000","last_peered":"2024-03-22T19:09:50.229345+0000","last_clean":"2024-03-22T19:09:50.229345+0000","last_became_active":"2024-03-22T19:09:13.653156+0000","last_became_peered":"2024-03-22T19:09:13.653156+0000","last_unstale":"2024-03-22T19:09:50.229345+0000","last_undegraded":"2024-03-22T19:09:50.229345+0000","last_fullsized":"2024-03-22T19:09:50.229345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:50:43.746165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.412328+0000","last_change":"2024-03-22T19:09:51.412328+0000","last_active":"2024-03-22T19:09:51.412328+0000","last_peered":"2024-03-22T19:09:51.412328+0000","last_clean":"2024-03-22T19:09:51.412328+0000","last_became_active":"2024-03-22T19:09:51.411942+0000","last_became_peered":"2024-03-22T19:09:51.411942+0000","last_unstale":"2024-03-22T19:09:51.412328+0000","last_undegraded":"2024-03-22T19:09:51.412328+0000","last_fullsized":"2024-03-22T19:09:51.412328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T00:39:48.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225949+0000","last_change":"2024-03-22T19:09:13.656807+0000","last_active":"2024-03-22T19:09:50.225949+0000","last_peered":"2024-03-22T19:09:50.225949+0000","last_clean":"2024-03-22T19:09:50.225949+0000","last_became_active":"2024-03-22T19:09:13.656498+0000","last_became_peered":"2024-03-22T19:09:13.656498+0000","last_unstale":"2024-03-22T19:09:50.225949+0000","last_undegraded":"2024-03-22T19:09:50.225949+0000","last_fullsized":"2024-03-22T19:09:50.225949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:31:53.236140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.225646+0000","last_change":"2024-03-22T19:09:13.650524+0000","last_active":"2024-03-22T19:09:50.225646+0000","last_peered":"2024-03-22T19:09:50.225646+0000","last_clean":"2024-03-22T19:09:50.225646+0000","last_became_active":"2024-03-22T19:09:13.650120+0000","last_became_peered":"2024-03-22T19:09:13.650120+0000","last_unstale":"2024-03-22T19:09:50.225646+0000","last_undegraded":"2024-03-22T19:09:50.225646+0000","last_fullsized":"2024-03-22T19:09:50.225646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T05:52:45.699058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-22T19:09:51.280034+0000","last_change":"2024-03-22T19:09:51.280034+0000","last_active":"2024-03-22T19:09:51.280034+0000","last_peered":"2024-03-22T19:09:51.280034+0000","last_clean":"2024-03-22T19:09:51.280034+0000","last_became_active":"2024-03-22T19:09:51.279604+0000","last_became_peered":"2024-03-22T19:09:51.279604+0000","last_unstale":"2024-03-22T19:09:51.280034+0000","last_undegraded":"2024-03-22T19:09:51.280034+0000","last_fullsized":"2024-03-22T19:09:51.280034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T03:38:59.518086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229596+0000","last_change":"2024-03-22T19:09:13.639475+0000","last_active":"2024-03-22T19:09:50.229596+0000","last_peered":"2024-03-22T19:09:50.229596+0000","last_clean":"2024-03-22T19:09:50.229596+0000","last_became_active":"2024-03-22T19:09:13.639130+0000","last_became_peered":"2024-03-22T19:09:13.639130+0000","last_unstale":"2024-03-22T19:09:50.229596+0000","last_undegraded":"2024-03-22T19:09:50.229596+0000","last_fullsized":"2024-03-22T19:09:50.229596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T21:39:58.868871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.226600+0000","last_change":"2024-03-22T19:09:13.653982+0000","last_active":"2024-03-22T19:09:50.226600+0000","last_peered":"2024-03-22T19:09:50.226600+0000","last_clean":"2024-03-22T19:09:50.226600+0000","last_became_active":"2024-03-22T19:09:13.653368+0000","last_became_peered":"2024-03-22T19:09:13.653368+0000","last_unstale":"2024-03-22T19:09:50.226600+0000","last_undegraded":"2024-03-22T19:09:50.226600+0000","last_fullsized":"2024-03-22T19:09:50.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T02:24:23.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230693+0000","last_change":"2024-03-22T19:09:13.660398+0000","last_active":"2024-03-22T19:09:50.230693+0000","last_peered":"2024-03-22T19:09:50.230693+0000","last_clean":"2024-03-22T19:09:50.230693+0000","last_became_active":"2024-03-22T19:09:13.659133+0000","last_became_peered":"2024-03-22T19:09:13.659133+0000","last_unstale":"2024-03-22T19:09:50.230693+0000","last_undegraded":"2024-03-22T19:09:50.230693+0000","last_fullsized":"2024-03-22T19:09:50.230693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:12:01.308268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229186+0000","last_change":"2024-03-22T19:09:13.644922+0000","last_active":"2024-03-22T19:09:50.229186+0000","last_peered":"2024-03-22T19:09:50.229186+0000","last_clean":"2024-03-22T19:09:50.229186+0000","last_became_active":"2024-03-22T19:09:13.644156+0000","last_became_peered":"2024-03-22T19:09:13.644156+0000","last_unstale":"2024-03-22T19:09:50.229186+0000","last_undegraded":"2024-03-22T19:09:50.229186+0000","last_fullsized":"2024-03-22T19:09:50.229186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T22:55:01.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.230140+0000","last_change":"2024-03-22T19:09:13.646961+0000","last_active":"2024-03-22T19:09:50.230140+0000","last_peered":"2024-03-22T19:09:50.230140+0000","last_clean":"2024-03-22T19:09:50.230140+0000","last_became_active":"2024-03-22T19:09:13.646630+0000","last_became_peered":"2024-03-22T19:09:13.646630+0000","last_unstale":"2024-03-22T19:09:50.230140+0000","last_undegraded":"2024-03-22T19:09:50.230140+0000","last_fullsized":"2024-03-22T19:09:50.230140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T04:16:20.931156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.228785+0000","last_change":"2024-03-22T19:09:13.646737+0000","last_active":"2024-03-22T19:09:50.228785+0000","last_peered":"2024-03-22T19:09:50.228785+0000","last_clean":"2024-03-22T19:09:50.228785+0000","last_became_active":"2024-03-22T19:09:13.646414+0000","last_became_peered":"2024-03-22T19:09:13.646414+0000","last_unstale":"2024-03-22T19:09:50.228785+0000","last_undegraded":"2024-03-22T19:09:50.228785+0000","last_fullsized":"2024-03-22T19:09:50.228785+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T01:14:07.114878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.221650+0000","last_change":"2024-03-22T19:09:13.631206+0000","last_active":"2024-03-22T19:09:50.221650+0000","last_peered":"2024-03-22T19:09:50.221650+0000","last_clean":"2024-03-22T19:09:50.221650+0000","last_became_active":"2024-03-22T19:09:13.630885+0000","last_became_peered":"2024-03-22T19:09:13.630885+0000","last_unstale":"2024-03-22T19:09:50.221650+0000","last_undegraded":"2024-03-22T19:09:50.221650+0000","last_fullsized":"2024-03-22T19:09:50.221650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T06:46:58.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-22T19:09:50.229710+0000","last_change":"2024-03-22T19:09:13.649441+0000","last_active":"2024-03-22T19:09:50.229710+0000","last_peered":"2024-03-22T19:09:50.229710+0000","last_clean":"2024-03-22T19:09:50.229710+0000","last_became_active":"2024-03-22T19:09:13.649126+0000","last_became_peered":"2024-03-22T19:09:13.649126+0000","last_unstale":"2024-03-22T19:09:50.229710+0000","last_undegraded":"2024-03-22T19:09:50.229710+0000","last_fullsized":"2024-03-22T19:09:50.229710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T19:09:13.115558+0000","last_clean_scrub_stamp":"2024-03-22T19:09:13.115558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T19:33:31.850253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246285,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66581,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":890,"num_read_kb":519,"num_write":233,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199743,"data_compressed":5466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":430423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95299999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.498,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82299999999999995,"5min":1.109,"15min":1.109},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:09:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.54500000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.032,"5min":1.214,"15min":1.214},"last":0.752},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Mar 22 19:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78600000000000003,"15min":0.79200000000000004},"min":{"1min":0.53600000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89000000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.63,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.73099999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.63800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":1.355,"15min":1.355},"last":0.74299999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1080662,"data_compressed":37475,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57099999999999995,"5min":0.628,"15min":0.628},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.58499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.624}]},{"osd":2,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.69099999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69099999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77200000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69799999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80600000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.749},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.875,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90000000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.58399999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86799999999999999,"5min":1.159,"15min":1.159},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.55800000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.81200000000000006}]},{"osd":11,"last update":"Fri Mar 22 19:10:40 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.63200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.61199999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82699999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":430289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.76500000000000001,"15min":0.78500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80200000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.70899999999999996,"15min":0.72099999999999997},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.048,"15min":1.048},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.998,"15min":0.998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.77000000000000002,"15min":0.80600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81299999999999994,"5min":1.355,"15min":1.355},"last":0.623}]},{"osd":5,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.87,"15min":0.89800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88200000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.504,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84199999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.89900000000000002,"15min":0.94199999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.86099999999999999,"15min":0.89300000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.87,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.84999999999999998,"15min":0.876},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96799999999999997,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Mar 22 19:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.88200000000000001,"15min":0.90700000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0069999999999999,"5min":1.476,"15min":1.476},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.94699999999999995,"15min":0.98099999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Mar 22 19:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98199999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.90100000000000002,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.86699999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.875,"15min":0.875},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.63500000000000001},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:09:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.60599999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.755,"15min":0.72499999999999998},"min":{"1min":0.66100000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.88600000000000001}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.56000000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Fri Mar 22 19:10:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.745},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.80000000000000004}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":1023067,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.438},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.35399999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.35899999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69499999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.436,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.50800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83299999999999996,"5min":1.111,"15min":1.111},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.496,"15min":0.496},"min":{"1min":0.40899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.75700000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79100000000000004,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.56000000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88400000000000001,"5min":1.155,"15min":1.155},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.73599999999999999,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"last":0.57599999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:10:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.878,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.499},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.56299999999999994,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91000000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.626}]},{"osd":9,"last update":"Fri Mar 22 19:10:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92100000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.044,"15min":1.044},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri Mar 22 19:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.63700000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.60399999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73299999999999998}]},{"osd":11,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71699999999999997}]}]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":430264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0960000000000001,"5min":1.369,"15min":1.369},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.496},{"interface":"front","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.371,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.131,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.497,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.56000000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.73199999999999998}]},{"osd":8,"last update":"Fri Mar 22 19:09:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.82299999999999995,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.87,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.68600000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.77900000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.626},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Mar 22 19:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82899999999999996,"15min":0.82699999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":490521,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:40 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.7320000000000002,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.76100000000000001},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":6.71,"15min":6.71},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Mar 22 19:09:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.48199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83699999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.379},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47999999999999998,"15min":0.442},"min":{"1min":0.35699999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.874,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86499999999999999,"5min":0.88,"15min":0.88},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Mar 22 19:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.441,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82399999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60199999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.77200000000000002,"15min":0.752},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.749},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67900000000000005}]},{"osd":10,"last update":"Fri Mar 22 19:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Mar 22 19:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":431297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 19:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.494,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.748}]},{"osd":3,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81499999999999995,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.63400000000000001},"min":{"1min":0.49099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64400000000000002,"15min":0.61199999999999999},"min":{"1min":0.501,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Mar 22 19:10:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.78200000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:10:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84999999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.67900000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1919999999999999,"5min":1.266,"15min":1.266},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.63700000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":489976,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.50600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.62}]},{"osd":1,"last update":"Fri Mar 22 19:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Mar 22 19:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70699999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Mar 22 19:09:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.437},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65300000000000002,"5min":1.448,"15min":1.448},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Mar 22 19:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.876,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Mar 22 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Mar 22 19:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.60299999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.53200000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.69899999999999995}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.54800000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69699999999999995,"15min":0.68500000000000005},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.002},"last":0.80200000000000005}]},{"osd":11,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72899999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84099999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.46200000000000002,"15min":0.432},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.56299999999999994},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2250000000000001,"5min":1.244,"15min":1.244},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0069999999999999,"5min":1.367,"15min":1.367},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84599999999999997,"5min":1.208,"15min":1.208},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Mar 22 19:09:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.417,"15min":1.417},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97599999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.628}]},{"osd":6,"last update":"Fri Mar 22 19:10:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83099999999999996,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.069,"5min":1.641,"15min":1.641},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Mar 22 19:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1279999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97899999999999998,"5min":1.163,"15min":1.163},"last":0.70199999999999996}]},{"osd":8,"last update":"Fri Mar 22 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":0.87},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0800000000000001,"5min":1.355,"15min":1.355},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91500000000000004,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1639999999999999,"5min":1.331,"15min":1.331},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0229999999999999,"5min":1.302,"15min":1.302},"last":0.76300000000000001}]},{"osd":11,"last update":"Fri Mar 22 19:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:08:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.33800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58299999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.38300000000000001},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.56200000000000006,"5min":0.629,"15min":0.629},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.45300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80900000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.379},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.44,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Mar 22 19:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.434}]},{"osd":5,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83599999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.65600000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Mar 22 19:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.504}]},{"osd":8,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60099999999999998}]},{"osd":9,"last update":"Fri Mar 22 19:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":1.214,"15min":1.214},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Mar 22 19:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93899999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.01,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.65200000000000002}]},{"osd":11,"last update":"Fri Mar 22 19:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70899999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1022485,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67600000000000005,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85799999999999998,"5min":3.887,"15min":3.887},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.66200000000000003,"15min":0.71299999999999997},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38,"15min":0.37},"min":{"1min":0.32800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.371},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Mar 22 19:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.497}]},{"osd":6,"last update":"Fri Mar 22 19:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.72899999999999998,"15min":0.77500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.749,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.621},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Mar 22 19:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Mar 22 19:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.751},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.45100000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Mar 22 19:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Mar 22 19:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.51200000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34010,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35653,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61663,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61809,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61341,"data_compressed":1822,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T19:10:49.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 -- 172.21.15.86:0/2708952138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 msgr2=0x7f7358101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 --2- 172.21.15.86:0/2708952138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358101070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f734c0099c0 tx=0x7f734c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 -- 172.21.15.86:0/2708952138 shutdown_connections 2024-03-22T19:10:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 --2- 172.21.15.86:0/2708952138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 0x7f7358109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 --2- 172.21.15.86:0/2708952138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358101070 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 --2- 172.21.15.86:0/2708952138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7358101e40 0x7f73581022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 -- 172.21.15.86:0/2708952138 >> 172.21.15.86:0/2708952138 conn(0x7f73580fc1c0 msgr2=0x7f73580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 -- 172.21.15.86:0/2708952138 shutdown_connections 2024-03-22T19:10:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.027+0000 7f7360800700 1 -- 172.21.15.86:0/2708952138 wait complete. 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.028+0000 7f7360800700 1 Processor -- start 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.028+0000 7f7360800700 1 -- start start 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f7360800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f7360800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7358101e40 0x7f7358197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f7360800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 0x7f735819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f7360800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735819e730 con 0x7f7358100c50 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f7360800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735819e870 con 0x7f73581027e0 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f7360800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735819ea00 con 0x7f7358101e40 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f735de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f735de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43000/0 (socket says 172.21.15.86:43000) 2024-03-22T19:10:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f735de00700 1 -- 172.21.15.86:0/3721064265 learned_addr learned my addr 172.21.15.86:0/3721064265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f735e800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 0x7f735819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.029+0000 7f735d400700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7358101e40 0x7f7358197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735de00700 1 -- 172.21.15.86:0/3721064265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7358101e40 msgr2=0x7f7358197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735de00700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7358101e40 0x7f7358197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735de00700 1 -- 172.21.15.86:0/3721064265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 msgr2=0x7f735819e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735de00700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 0x7f735819e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735de00700 1 -- 172.21.15.86:0/3721064265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0095f0 con 0x7f7358100c50 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735e800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 0x7f735819e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f735de00700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358196c00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f734c003730 tx=0x7f734c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c01d070 con 0x7f7358100c50 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734c00bda0 con 0x7f7358100c50 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c004b30 con 0x7f7358100c50 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735819ec80 con 0x7f7358100c50 2024-03-22T19:10:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.030+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735819f050 con 0x7f7358100c50 2024-03-22T19:10:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.031+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7358108120 con 0x7f7358100c50 2024-03-22T19:10:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.033+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135439+0+0 (secure 0 0 0) 0x7f734c004c90 con 0x7f7358100c50 2024-03-22T19:10:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.035+0000 7f7352a00700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f73400a0ba0 0x7f73400a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.035+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f734c0ccf90 con 0x7f7358100c50 2024-03-22T19:10:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.035+0000 7f735d400700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f73400a0ba0 0x7f73400a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.036+0000 7f735d400700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f73400a0ba0 0x7f73400a3060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7348005950 tx=0x7f734800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.039+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f734c096880 con 0x7f7358100c50 2024-03-22T19:10:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.289+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f734c096c50 con 0x7f7358100c50 2024-03-22T19:10:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.319+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73580679e0 con 0x7f7358100c50 2024-03-22T19:10:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.321+0000 7f7352a00700 1 -- 172.21.15.86:0/3721064265 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7f734c09a310 con 0x7f7358100c50 2024-03-22T19:10:50.322 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:50.323 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:50.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f73400a0ba0 msgr2=0x7f73400a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f73400a0ba0 0x7f73400a3060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7348005950 tx=0x7f734800b410 comp rx=0 tx=0).stop 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 msgr2=0x7f7358196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358196c00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f734c003730 tx=0x7f734c00fa00 comp rx=0 tx=0).stop 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 shutdown_connections 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f73400a0ba0 0x7f73400a3060 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73581027e0 0x7f735819e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7358100c50 0x7f7358196c00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 --2- 172.21.15.86:0/3721064265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7358101e40 0x7f7358197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 >> 172.21.15.86:0/3721064265 conn(0x7f73580fc1c0 msgr2=0x7f7358105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 shutdown_connections 2024-03-22T19:10:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.323+0000 7f7360800700 1 -- 172.21.15.86:0/3721064265 wait complete. 2024-03-22T19:10:50.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:10:50.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:10:50.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:50 smithi086 ceph-mon[38032]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:50.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:50 smithi086 ceph-mon[38032]: from='client.25322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:50.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 -- 172.21.15.86:0/3101611090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 msgr2=0x7f3d60101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 --2- 172.21.15.86:0/3101611090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60101160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009920 tx=0x7f3d50009c30 comp rx=0 tx=0).stop 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 -- 172.21.15.86:0/3101611090 shutdown_connections 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 --2- 172.21.15.86:0/3101611090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d601028d0 0x7f3d60109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 --2- 172.21.15.86:0/3101611090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60101160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 --2- 172.21.15.86:0/3101611090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d60101f30 0x7f3d60102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 -- 172.21.15.86:0/3101611090 >> 172.21.15.86:0/3101611090 conn(0x7f3d600fc2b0 msgr2=0x7f3d600fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 -- 172.21.15.86:0/3101611090 shutdown_connections 2024-03-22T19:10:50.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.538+0000 7f3d66200700 1 -- 172.21.15.86:0/3101611090 wait complete. 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 Processor -- start 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 -- start start 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60101f30 0x7f3d60199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 0x7f3d601a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d601a0b60 con 0x7f3d601028d0 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d601a0ca0 con 0x7f3d60100d40 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.539+0000 7f3d66200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d601a0e30 con 0x7f3d60101f30 2024-03-22T19:10:50.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 0x7f3d601a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60101f30 0x7f3d60199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 0x7f3d601a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43020/0 (socket says 172.21.15.86:43020) 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 -- 172.21.15.86:0/3589475243 learned_addr learned my addr 172.21.15.86:0/3589475243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53054/0 (socket says 172.21.15.86:53054) 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60101f30 0x7f3d60199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:35294/0 (socket says 172.21.15.86:35294) 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 -- 172.21.15.86:0/3589475243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60101f30 msgr2=0x7f3d60199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60101f30 0x7f3d60199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 -- 172.21.15.86:0/3589475243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 msgr2=0x7f3d60199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 -- 172.21.15.86:0/3589475243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0095f0 con 0x7f3d601028d0 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.540+0000 7f3d5fe00700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 0x7f3d601a0590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800ef30 tx=0x7f3d5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:50.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.541+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58018070 con 0x7f3d601028d0 2024-03-22T19:10:50.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.541+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d50009520 con 0x7f3d601028d0 2024-03-22T19:10:50.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.541+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5800f040 con 0x7f3d601028d0 2024-03-22T19:10:50.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.541+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800cca0 con 0x7f3d601028d0 2024-03-22T19:10:50.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.541+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d601a1480 con 0x7f3d601028d0 2024-03-22T19:10:50.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.542+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d60067ac0 con 0x7f3d601028d0 2024-03-22T19:10:50.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.544+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f3d58003a00 con 0x7f3d601028d0 2024-03-22T19:10:50.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.545+0000 7f3d57e00700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3d440a0a90 0x7f3d440a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.545+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3d580cbb00 con 0x7f3d601028d0 2024-03-22T19:10:50.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.545+0000 7f3d5f400700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3d440a0a90 0x7f3d440a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.546+0000 7f3d5f400700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3d440a0a90 0x7f3d440a2f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d500098f0 tx=0x7f3d50019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:50.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.549+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3d580cde60 con 0x7f3d601028d0 2024-03-22T19:10:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:50 smithi167 ceph-mon[47728]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:50 smithi167 ceph-mon[47728]: from='client.25322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:50 smithi131 ceph-mon[78559]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:50.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:50 smithi131 ceph-mon[78559]: from='client.25322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.754+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d601a16a0 con 0x7f3d601028d0 2024-03-22T19:10:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.754+0000 7f3d57e00700 1 -- 172.21.15.86:0/3589475243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d58095550 con 0x7f3d601028d0 2024-03-22T19:10:50.756 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.756+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3d440a0a90 msgr2=0x7f3d440a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.756+0000 7f3d66200700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3d440a0a90 0x7f3d440a2f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d500098f0 tx=0x7f3d50019040 comp rx=0 tx=0).stop 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.756+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 msgr2=0x7f3d601a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.756+0000 7f3d66200700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 0x7f3d601a0590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800ef30 tx=0x7f3d5800c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 shutdown_connections 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3d440a0a90 0x7f3d440a2f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d601028d0 0x7f3d601a0590 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d60100d40 0x7f3d60199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 --2- 172.21.15.86:0/3589475243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60101f30 0x7f3d60199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 >> 172.21.15.86:0/3589475243 conn(0x7f3d600fc2b0 msgr2=0x7f3d60107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 shutdown_connections 2024-03-22T19:10:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.757+0000 7f3d66200700 1 -- 172.21.15.86:0/3589475243 wait complete. 2024-03-22T19:10:50.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:10:50.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 -- 172.21.15.86:0/565205705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418100d40 msgr2=0x7fa418101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 --2- 172.21.15.86:0/565205705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418100d40 0x7fa418101160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa4080099c0 tx=0x7fa408009cd0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 -- 172.21.15.86:0/565205705 shutdown_connections 2024-03-22T19:10:50.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 --2- 172.21.15.86:0/565205705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4181028d0 0x7fa418109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 --2- 172.21.15.86:0/565205705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418100d40 0x7fa418101160 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 --2- 172.21.15.86:0/565205705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418101f30 0x7fa418102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 -- 172.21.15.86:0/565205705 >> 172.21.15.86:0/565205705 conn(0x7fa4180fc2b0 msgr2=0x7fa4180fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 -- 172.21.15.86:0/565205705 shutdown_connections 2024-03-22T19:10:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.970+0000 7fa41e400700 1 -- 172.21.15.86:0/565205705 wait complete. 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.971+0000 7fa41e400700 1 Processor -- start 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.971+0000 7fa41e400700 1 -- start start 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa41e400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418100d40 0x7fa418198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa41e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa418101f30 0x7fa418199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa41e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 0x7fa4181a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa41e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181a09e0 con 0x7fa4181028d0 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa41e400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181a0b20 con 0x7fa418101f30 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa41e400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181a0cb0 con 0x7fa418100d40 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 0x7fa4181a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418100d40 0x7fa418198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 0x7fa4181a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43034/0 (socket says 172.21.15.86:43034) 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 -- 172.21.15.86:0/594729884 learned_addr learned my addr 172.21.15.86:0/594729884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa416a00700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa418101f30 0x7fa418199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 -- 172.21.15.86:0/594729884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418100d40 msgr2=0x7fa418198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418100d40 0x7fa418198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 -- 172.21.15.86:0/594729884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa418101f30 msgr2=0x7fa418199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa418101f30 0x7fa418199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417e00700 1 -- 172.21.15.86:0/594729884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080095f0 con 0x7fa4181028d0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.972+0000 7fa417400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418100d40 0x7fa418198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.973+0000 7fa417e00700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 0x7fa4181a0410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa41000ef30 tx=0x7fa41000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.973+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410018070 con 0x7fa4181028d0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.973+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41000f040 con 0x7fa4181028d0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.973+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4181a0f30 con 0x7fa4181028d0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.973+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000cd00 con 0x7fa4181028d0 2024-03-22T19:10:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.973+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4181a13b0 con 0x7fa4181028d0 2024-03-22T19:10:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.974+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa418067ac0 con 0x7fa4181028d0 2024-03-22T19:10:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.976+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa410003a00 con 0x7fa4181028d0 2024-03-22T19:10:50.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.977+0000 7fa40fe00700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3fc0a0a90 0x7fa3fc0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:50.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.977+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa4100cbb40 con 0x7fa4181028d0 2024-03-22T19:10:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.977+0000 7fa417400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3fc0a0a90 0x7fa3fc0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.978+0000 7fa417400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3fc0a0a90 0x7fa3fc0a2f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa408003730 tx=0x7fa40800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:50.981+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa410095590 con 0x7fa4181028d0 2024-03-22T19:10:51.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.187+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4181a1690 con 0x7fa4181028d0 2024-03-22T19:10:51.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.188+0000 7fa40fe00700 1 -- 172.21.15.86:0/594729884 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa410093e60 con 0x7fa4181028d0 2024-03-22T19:10:51.189 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3fc0a0a90 msgr2=0x7fa3fc0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3fc0a0a90 0x7fa3fc0a2f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa408003730 tx=0x7fa40800b540 comp rx=0 tx=0).stop 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 msgr2=0x7fa4181a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 0x7fa4181a0410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa41000ef30 tx=0x7fa41000c3a0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 shutdown_connections 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3fc0a0a90 0x7fa3fc0a2f50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4181028d0 0x7fa4181a0410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa418100d40 0x7fa418198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 --2- 172.21.15.86:0/594729884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa418101f30 0x7fa418199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.190+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 >> 172.21.15.86:0/594729884 conn(0x7fa4180fc2b0 msgr2=0x7fa418072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:51.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.191+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 shutdown_connections 2024-03-22T19:10:51.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.191+0000 7fa41e400700 1 -- 172.21.15.86:0/594729884 wait complete. 2024-03-22T19:10:51.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 -- 172.21.15.86:0/2761375809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8201028d0 msgr2=0x7ff820109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 --2- 172.21.15.86:0/2761375809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8201028d0 0x7ff820109180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b0d0 tx=0x7ff81800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 -- 172.21.15.86:0/2761375809 shutdown_connections 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 --2- 172.21.15.86:0/2761375809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8201028d0 0x7ff820109180 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 --2- 172.21.15.86:0/2761375809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff820100d40 0x7ff820101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 --2- 172.21.15.86:0/2761375809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff820101f30 0x7ff820102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 -- 172.21.15.86:0/2761375809 >> 172.21.15.86:0/2761375809 conn(0x7ff8200fc2b0 msgr2=0x7ff8200fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 -- 172.21.15.86:0/2761375809 shutdown_connections 2024-03-22T19:10:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.405+0000 7ff825000700 1 -- 172.21.15.86:0/2761375809 wait complete. 2024-03-22T19:10:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 Processor -- start 2024-03-22T19:10:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 -- start start 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 0x7ff820199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff820101f30 0x7ff820199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8201028d0 0x7ff8201a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a0b70 con 0x7ff820100d40 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a0cb0 con 0x7ff8201028d0 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff825000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a0e40 con 0x7ff820101f30 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.407+0000 7ff81de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 0x7ff820199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 0x7ff820199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43064/0 (socket says 172.21.15.86:43064) 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 -- 172.21.15.86:0/3279069458 learned_addr learned my addr 172.21.15.86:0/3279069458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81d400700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff820101f30 0x7ff820199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81e800700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8201028d0 0x7ff8201a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 -- 172.21.15.86:0/3279069458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff820101f30 msgr2=0x7ff820199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff820101f30 0x7ff820199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 -- 172.21.15.86:0/3279069458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8201028d0 msgr2=0x7ff8201a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8201028d0 0x7ff8201a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 -- 172.21.15.86:0/3279069458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818009d00 con 0x7ff820100d40 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81e800700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8201028d0 0x7ff8201a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff81de00700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 0x7ff820199040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff81000e970 tx=0x7ff81000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.408+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810009790 con 0x7ff820100d40 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.409+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81000eec0 con 0x7ff820100d40 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.409+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8201a1120 con 0x7ff820100d40 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.409+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810003cf0 con 0x7ff820100d40 2024-03-22T19:10:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.409+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201a1630 con 0x7ff820100d40 2024-03-22T19:10:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.410+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff820067ac0 con 0x7ff820100d40 2024-03-22T19:10:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.411+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff81000ccd0 con 0x7ff820100d40 2024-03-22T19:10:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.413+0000 7ff816a00700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8040a0a90 0x7ff8040a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.413+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff810014070 con 0x7ff820100d40 2024-03-22T19:10:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.413+0000 7ff81d400700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8040a0a90 0x7ff8040a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.414+0000 7ff81d400700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8040a0a90 0x7ff8040a2f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009c60 tx=0x7ff80c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:51.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.417+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff810095c80 con 0x7ff820100d40 2024-03-22T19:10:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:51 smithi086 ceph-mon[38032]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:51 smithi086 ceph-mon[38032]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-22T19:10:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:51 smithi086 ceph-mon[38032]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-22T19:10:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:51 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3721064265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:51 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3589475243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:51.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:51 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/594729884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:51 smithi167 ceph-mon[47728]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:51 smithi167 ceph-mon[47728]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-22T19:10:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:51 smithi167 ceph-mon[47728]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-22T19:10:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:51 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3721064265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:51 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3589475243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:51 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/594729884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.626+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8201a19a0 con 0x7ff820100d40 2024-03-22T19:10:51.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.627+0000 7ff816a00700 1 -- 172.21.15.86:0/3279069458 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff8100992a0 con 0x7ff820100d40 2024-03-22T19:10:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:10:51.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8040a0a90 msgr2=0x7ff8040a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8040a0a90 0x7ff8040a2f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009c60 tx=0x7ff80c009260 comp rx=0 tx=0).stop 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 msgr2=0x7ff820199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 0x7ff820199040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff81000e970 tx=0x7ff81000ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 shutdown_connections 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8040a0a90 0x7ff8040a2f50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8201028d0 0x7ff8201a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff820100d40 0x7ff820199040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 --2- 172.21.15.86:0/3279069458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff820101f30 0x7ff820199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 >> 172.21.15.86:0/3279069458 conn(0x7ff8200fc2b0 msgr2=0x7ff8200fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 shutdown_connections 2024-03-22T19:10:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.629+0000 7ff825000700 1 -- 172.21.15.86:0/3279069458 wait complete. 2024-03-22T19:10:51.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:10:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:51 smithi131 ceph-mon[78559]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T19:10:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:51 smithi131 ceph-mon[78559]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-22T19:10:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:51 smithi131 ceph-mon[78559]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-22T19:10:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:51 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3721064265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:51 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3589475243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:10:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:51 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/594729884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:10:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 -- 172.21.15.86:0/503641940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c100c50 msgr2=0x7f1b2c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 --2- 172.21.15.86:0/503641940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c100c50 0x7f1b2c101070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009920 tx=0x7f1b1c009c30 comp rx=0 tx=0).stop 2024-03-22T19:10:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 -- 172.21.15.86:0/503641940 shutdown_connections 2024-03-22T19:10:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 --2- 172.21.15.86:0/503641940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c1027e0 0x7f1b2c109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 --2- 172.21.15.86:0/503641940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c100c50 0x7f1b2c101070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 --2- 172.21.15.86:0/503641940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b2c101e40 0x7f1b2c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 -- 172.21.15.86:0/503641940 >> 172.21.15.86:0/503641940 conn(0x7f1b2c0fc1c0 msgr2=0x7f1b2c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 -- 172.21.15.86:0/503641940 shutdown_connections 2024-03-22T19:10:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.841+0000 7f1b32400700 1 -- 172.21.15.86:0/503641940 wait complete. 2024-03-22T19:10:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.842+0000 7f1b32400700 1 Processor -- start 2024-03-22T19:10:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.842+0000 7f1b32400700 1 -- start start 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b32400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 0x7f1b2c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b32400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c101e40 0x7f1b2c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b32400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b2c1027e0 0x7f1b2c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b32400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c1a09f0 con 0x7f1b2c100c50 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b32400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c1a0b30 con 0x7f1b2c101e40 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b32400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c1a0cc0 con 0x7f1b2c1027e0 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 0x7f1b2c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 0x7f1b2c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43088/0 (socket says 172.21.15.86:43088) 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 -- 172.21.15.86:0/1120476240 learned_addr learned my addr 172.21.15.86:0/1120476240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b23e00700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c101e40 0x7f1b2c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2be00700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b2c1027e0 0x7f1b2c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 -- 172.21.15.86:0/1120476240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b2c1027e0 msgr2=0x7f1b2c1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b2c1027e0 0x7f1b2c1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 -- 172.21.15.86:0/1120476240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c101e40 msgr2=0x7f1b2c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c101e40 0x7f1b2c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.843+0000 7f1b2b400700 1 -- 172.21.15.86:0/1120476240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c009520 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.844+0000 7f1b2b400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 0x7f1b2c198f10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009230 tx=0x7f1b1c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.844+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c01c070 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.844+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b1c00bba0 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.844+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c1a0f40 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.844+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0049c0 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.844+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c1a12b0 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.845+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c0679e0 con 0x7f1b2c100c50 2024-03-22T19:10:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.847+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f1b1c020070 con 0x7f1b2c100c50 2024-03-22T19:10:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.849+0000 7f1b28c00700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b100a0a90 0x7f1b100a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.849+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1b1c0ccd50 con 0x7f1b2c100c50 2024-03-22T19:10:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.849+0000 7f1b23e00700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b100a0a90 0x7f1b100a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:51.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.850+0000 7f1b23e00700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b100a0a90 0x7f1b100a2f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009c60 tx=0x7f1b14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:51.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:51.853+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1b1c0967a0 con 0x7f1b2c100c50 2024-03-22T19:10:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.103+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b2c1a15f0 con 0x7f1b2c100c50 2024-03-22T19:10:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.103+0000 7f1b28c00700 1 -- 172.21.15.86:0/1120476240 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f1b1c099dc0 con 0x7f1b2c100c50 2024-03-22T19:10:52.105 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:52.105 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:10:44.254849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:10:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b100a0a90 msgr2=0x7f1b100a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b100a0a90 0x7f1b100a2f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009c60 tx=0x7f1b14009260 comp rx=0 tx=0).stop 2024-03-22T19:10:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 msgr2=0x7f1b2c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 0x7f1b2c198f10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009230 tx=0x7f1b1c00bef0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 shutdown_connections 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1b100a0a90 0x7f1b100a2f50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b2c1027e0 0x7f1b2c1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c100c50 0x7f1b2c198f10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 --2- 172.21.15.86:0/1120476240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c101e40 0x7f1b2c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 >> 172.21.15.86:0/1120476240 conn(0x7f1b2c0fc1c0 msgr2=0x7f1b2c107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 shutdown_connections 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.106+0000 7f1b32400700 1 -- 172.21.15.86:0/1120476240 wait complete. 2024-03-22T19:10:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-22T19:10:52.126 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 -- 172.21.15.86:0/2258451766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c108b00 msgr2=0x7fbe5c10af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 --2- 172.21.15.86:0/2258451766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c108b00 0x7fbe5c10af90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400b0a0 tx=0x7fbe5400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 -- 172.21.15.86:0/2258451766 shutdown_connections 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 --2- 172.21.15.86:0/2258451766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c108b00 0x7fbe5c10af90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 --2- 172.21.15.86:0/2258451766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c074df0 0x7fbe5c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 --2- 172.21.15.86:0/2258451766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe5c073810 0x7fbe5c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 -- 172.21.15.86:0/2258451766 >> 172.21.15.86:0/2258451766 conn(0x7fbe5c0fc1b0 msgr2=0x7fbe5c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 -- 172.21.15.86:0/2258451766 shutdown_connections 2024-03-22T19:10:52.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.321+0000 7fbe62400700 1 -- 172.21.15.86:0/2258451766 wait complete. 2024-03-22T19:10:52.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 Processor -- start 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 -- start start 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe5c073810 0x7fbe5c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 0x7fbe5c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c108b00 0x7fbe5c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c1a0a10 con 0x7fbe5c074df0 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c1a0b50 con 0x7fbe5c073810 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.322+0000 7fbe62400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c1a0ce0 con 0x7fbe5c108b00 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 0x7fbe5c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 0x7fbe5c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43106/0 (socket says 172.21.15.86:43106) 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 -- 172.21.15.86:0/3815519791 learned_addr learned my addr 172.21.15.86:0/3815519791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe5b400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe5c073810 0x7fbe5c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe5be00700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c108b00 0x7fbe5c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 -- 172.21.15.86:0/3815519791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c108b00 msgr2=0x7fbe5c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c108b00 0x7fbe5c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 -- 172.21.15.86:0/3815519791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe5c073810 msgr2=0x7fbe5c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe5c073810 0x7fbe5c198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 -- 172.21.15.86:0/3815519791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44009520 con 0x7fbe5c074df0 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.323+0000 7fbe53e00700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 0x7fbe5c199420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400ea50 tx=0x7fbe4400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.324+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400cbd0 con 0x7fbe5c074df0 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.324+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4400cd30 con 0x7fbe5c074df0 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.324+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe54009d00 con 0x7fbe5c074df0 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.324+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44018680 con 0x7fbe5c074df0 2024-03-22T19:10:52.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.324+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c1a12b0 con 0x7fbe5c074df0 2024-03-22T19:10:52.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.325+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c1078c0 con 0x7fbe5c074df0 2024-03-22T19:10:52.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.327+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fbe44010430 con 0x7fbe5c074df0 2024-03-22T19:10:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.329+0000 7fbe58c00700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbe400a0a90 0x7fbe400a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.329+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fbe44014070 con 0x7fbe5c074df0 2024-03-22T19:10:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.329+0000 7fbe5b400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbe400a0a90 0x7fbe400a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.330+0000 7fbe5b400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbe400a0a90 0x7fbe400a2f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009c60 tx=0x7fbe4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.333+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbe44096030 con 0x7fbe5c074df0 2024-03-22T19:10:52.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:52 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3279069458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:52.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:52 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1120476240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:52 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3279069458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:52 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1120476240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:52.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.577+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe5c067e30 con 0x7fbe5c074df0 2024-03-22T19:10:52.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.579+0000 7fbe58c00700 1 -- 172.21.15.86:0/3815519791 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7fbe44099650 con 0x7fbe5c074df0 2024-03-22T19:10:52.580 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:10:52.580 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbe400a0a90 msgr2=0x7fbe400a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbe400a0a90 0x7fbe400a2f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009c60 tx=0x7fbe4c009260 comp rx=0 tx=0).stop 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 msgr2=0x7fbe5c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 0x7fbe5c199420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400ea50 tx=0x7fbe4400ed60 comp rx=0 tx=0).stop 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 shutdown_connections 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbe400a0a90 0x7fbe400a2f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c108b00 0x7fbe5c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe5c073810 0x7fbe5c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 --2- 172.21.15.86:0/3815519791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5c074df0 0x7fbe5c199420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 >> 172.21.15.86:0/3815519791 conn(0x7fbe5c0fc1b0 msgr2=0x7fbe5c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 shutdown_connections 2024-03-22T19:10:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.581+0000 7fbe62400700 1 -- 172.21.15.86:0/3815519791 wait complete. 2024-03-22T19:10:52.603 DEBUG:tasks.fwd_scrub:Yielding 2024-03-22T19:10:52.603 INFO:teuthology.run_tasks:Running task workunit... 2024-03-22T19:10:52.616 INFO:tasks.workunit:Pulling workunits from ref 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:10:52.616 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-22T19:10:52.617 INFO:tasks.workunit:timeout=3h 2024-03-22T19:10:52.617 INFO:tasks.workunit:cleanup=True 2024-03-22T19:10:52.617 DEBUG:teuthology.orchestra.run.smithi086:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:52.617 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-22T19:10:52.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:10:52.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:10:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 7ah/122d Inode: 1099511628283 Links: 2 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:cephfs_t:s0 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 19:09:21.105682781 +0000 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 19:09:21.105682781 +0000 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 19:10:42.538553774 +0000 2024-03-22T19:10:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 19:09:21.105682781 +0000 2024-03-22T19:10:52.662 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-22T19:10:52.663 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-22T19:10:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:52 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3279069458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:10:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:52 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1120476240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:10:52.785 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 2024-03-22T19:10:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.827+0000 7f697be00700 1 -- 172.21.15.86:0/3525631309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c074df0 msgr2=0x7f697c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.827+0000 7f697be00700 1 --2- 172.21.15.86:0/3525631309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c074df0 0x7f697c073240 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f696c009990 tx=0x7f696c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 -- 172.21.15.86:0/3525631309 shutdown_connections 2024-03-22T19:10:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 --2- 172.21.15.86:0/3525631309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c108a50 0x7f697c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 --2- 172.21.15.86:0/3525631309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c074df0 0x7f697c073240 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 --2- 172.21.15.86:0/3525631309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c073810 0x7f697c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 -- 172.21.15.86:0/3525631309 >> 172.21.15.86:0/3525631309 conn(0x7f697c0fc0e0 msgr2=0x7f697c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 -- 172.21.15.86:0/3525631309 shutdown_connections 2024-03-22T19:10:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.828+0000 7f697be00700 1 -- 172.21.15.86:0/3525631309 wait complete. 2024-03-22T19:10:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.829+0000 7f697be00700 1 Processor -- start 2024-03-22T19:10:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.829+0000 7f697be00700 1 -- start start 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c073810 0x7f697c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 0x7f697c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c108a50 0x7f697c1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1a07b0 con 0x7f697c108a50 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697be00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1a0920 con 0x7f697c073810 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697be00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1a0ae0 con 0x7f697c074df0 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c108a50 0x7f697c1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 0x7f697c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 0x7f697c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:35378/0 (socket says 172.21.15.86:35378) 2024-03-22T19:10:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697a000700 1 -- 172.21.15.86:0/3530913705 learned_addr learned my addr 172.21.15.86:0/3530913705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c108a50 0x7f697c1a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43130/0 (socket says 172.21.15.86:43130) 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697aa00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c073810 0x7f697c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697a000700 1 -- 172.21.15.86:0/3530913705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c073810 msgr2=0x7f697c198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.830+0000 7f697a000700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c073810 0x7f697c198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f697a000700 1 -- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c108a50 msgr2=0x7f697c1a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f697a000700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c108a50 0x7f697c1a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f697a000700 1 -- 172.21.15.86:0/3530913705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0095f0 con 0x7f697c074df0 2024-03-22T19:10:52.833 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-22T19:10:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f697a000700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 0x7f697c199200 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f696800e970 tx=0x7f696800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f6973400700 1 -- 172.21.15.86:0/3530913705 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968009790 con 0x7f697c074df0 2024-03-22T19:10:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c1a0dc0 con 0x7f697c074df0 2024-03-22T19:10:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f6973400700 1 -- 172.21.15.86:0/3530913705 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696800c7e0 con 0x7f697c074df0 2024-03-22T19:10:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f6973400700 1 -- 172.21.15.86:0/3530913705 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968003c70 con 0x7f697c074df0 2024-03-22T19:10:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.831+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1a11f0 con 0x7f697c074df0 2024-03-22T19:10:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.836+0000 7f6973400700 1 -- 172.21.15.86:0/3530913705 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f696800cc50 con 0x7f697c074df0 2024-03-22T19:10:52.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.836+0000 7f6995545b80 1 Processor -- start 2024-03-22T19:10:52.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 -- start start 2024-03-22T19:10:52.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 0x55b9d6b27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6a7ea10 con 0x55b9d6b27a60 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6a7c3d0 con 0x55b9d6b270a0 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6995545b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6a77cc0 con 0x55b9d6b25e60 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6973400700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69600a0a90 0x7f69600a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f6973400700 1 -- 172.21.15.86:0/3530913705 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6968014070 con 0x7f697c074df0 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f697b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f697aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 0x55b9d6b27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f697aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 0x55b9d6b27520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53178/0 (socket says 172.21.15.86:53178) 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 -- 172.21.15.86:0/788295778 learned_addr learned my addr 172.21.15.86:0/788295778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.837+0000 7f697b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b262a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:35390/0 (socket says 172.21.15.86:35390) 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697a000700 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697b400700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69600a0a90 0x7f69600a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 -- 172.21.15.86:0/788295778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 msgr2=0x55b9d6b262a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b262a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 -- 172.21.15.86:0/788295778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 msgr2=0x55b9d6b2e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 -- 172.21.15.86:0/788295778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9d68f9430 con 0x55b9d6b270a0 2024-03-22T19:10:52.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697a000700 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:52.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697b400700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69600a0a90 0x7f69600a2f50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55b9d6b25c00 tx=0x7f697400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.838+0000 7f697aa00700 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 0x55b9d6b27520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f696c0098e0 tx=0x7f696c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e43fbe7718e7afa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6967e00700 1 -- 172.21.15.86:0/788295778 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c01d070 con 0x55b9d6b270a0 2024-03-22T19:10:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6967e00700 1 -- 172.21.15.86:0/788295778 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c01e040 con 0x55b9d6b270a0 2024-03-22T19:10:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6967e00700 1 -- 172.21.15.86:0/788295778 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00fe70 con 0x55b9d6b270a0 2024-03-22T19:10:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6995545b80 1 -- 172.21.15.86:0/788295778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 msgr2=0x55b9d6b27520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6995545b80 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 0x55b9d6b27520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f696c0098e0 tx=0x7f696c005ea0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6995545b80 1 -- 172.21.15.86:0/788295778 shutdown_connections 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6995545b80 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6995545b80 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b262a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.839+0000 7f6995545b80 1 --2- 172.21.15.86:0/788295778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b270a0 0x55b9d6b27520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.840+0000 7f6995545b80 1 -- 172.21.15.86:0/788295778 >> 172.21.15.86:0/788295778 conn(0x55b9d6a9b2a0 msgr2=0x55b9d6b20e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.840+0000 7f6995545b80 1 -- 172.21.15.86:0/788295778 shutdown_connections 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.840+0000 7f6995545b80 1 -- 172.21.15.86:0/788295778 wait complete. 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.840+0000 7f6995545b80 1 -- start start 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.840+0000 7f6995545b80 1 Processor -- start 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6a13e00 con 0x55b9d6b270a0 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6a13aa0 con 0x55b9d6b27a60 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d68fb720 con 0x55b9d6b25e60 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f6995545b80 10 client.0 _ll_register_callbacks cb 0x55b9d687f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f697aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f697aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:35394/0 (socket says 172.21.15.86:35394) 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f697aa00700 1 -- 172.21.15.86:0/3360960232 learned_addr learned my addr 172.21.15.86:0/3360960232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f697a000700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.841+0000 7f697b400700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697a000700 1 -- 172.21.15.86:0/3360960232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 msgr2=0x55b9d6b2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697a000700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697a000700 1 -- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 msgr2=0x55b9d6b37280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697a000700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b37280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697a000700 1 -- 172.21.15.86:0/3360960232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1a11f0 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697b400700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f697a000700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b30240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f69680995b0 tx=0x7f6968099750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aada611422d00839 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f6964c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f6964c00700 1 -- 172.21.15.86:0/3360960232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69680b7ea0 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f6964c00700 1 -- 172.21.15.86:0/3360960232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f69680a1d30 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9d6b18cb0 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.842+0000 7f6964c00700 1 -- 172.21.15.86:0/3360960232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69680b2260 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 10 client.15351 fetch_fsmap learned FSMap version 27 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f697c10bb00 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6964c00700 1 -- 172.21.15.86:0/3360960232 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6968063080 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6964c00700 1 -- 172.21.15.86:0/3360960232 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f69680a18a0 con 0x55b9d6b270a0 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:10:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:10:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 20 client.15351 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b3bb50 0x55b9d6b3df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 4 client.15351 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.843+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9d6a70780 con 0x55b9d6b3bb50 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.844+0000 7f697aa00700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b3bb50 0x55b9d6b3df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.844+0000 7f697aa00700 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b3bb50 0x55b9d6b3df70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bff9dc485d8851 server_cookie=6157dd1299880af1 in_seq=0 out_seq=0 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.844+0000 7f6964c00700 10 client.15351 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6964c00700 1 -- 172.21.15.86:0/3360960232 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b9d6a70780 con 0x55b9d6b3bb50 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6964c00700 10 client.15351 handle_command_reply: tid=0 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 client.15351 shutdown 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 msgr2=0x55b9d6b30240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b30240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f69680995b0 tx=0x7f6968099750 comp rx=0 tx=0).stop 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 shutdown_connections 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b3bb50 0x55b9d6b3df70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6964c00700 0 client.15351 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 --2- 172.21.15.86:0/3360960232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b30240 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 >> 172.21.15.86:0/3360960232 conn(0x55b9d6a9b2a0 msgr2=0x55b9d6b203d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 shutdown_connections 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.845+0000 7f6995545b80 1 -- 172.21.15.86:0/3360960232 wait complete. 2024-03-22T19:10:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.846+0000 7f6995545b80 20 client.15351 trim_cache size 0 max 16384 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 Processor -- start 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 -- start start 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b270a0 0x55b9d6b37240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6b22640 con 0x55b9d6b27a60 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6b1d7c0 con 0x55b9d6b25e60 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f6995545b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6b39010 con 0x55b9d6b270a0 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f697b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f697aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f697aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53198/0 (socket says 172.21.15.86:53198) 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f697b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43152/0 (socket says 172.21.15.86:43152) 2024-03-22T19:10:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.863+0000 7f697aa00700 1 -- 172.21.15.86:0/4118422160 learned_addr learned my addr 172.21.15.86:0/4118422160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697a000700 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b270a0 0x55b9d6b37240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697b400700 1 -- 172.21.15.86:0/4118422160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b270a0 msgr2=0x55b9d6b37240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697b400700 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b270a0 0x55b9d6b37240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697b400700 1 -- 172.21.15.86:0/4118422160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 msgr2=0x55b9d6b2f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697b400700 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697b400700 1 -- 172.21.15.86:0/4118422160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9d68a5be0 con 0x55b9d6b27a60 2024-03-22T19:10:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697aa00700 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697a000700 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b270a0 0x55b9d6b37240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f697b400700 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e9e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6974008fc0 tx=0x7f6974009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da37116941e8a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f6970c00700 1 -- 172.21.15.86:0/4118422160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400e070 con 0x55b9d6b27a60 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f6970c00700 1 -- 172.21.15.86:0/4118422160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6974023040 con 0x55b9d6b27a60 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.864+0000 7f6970c00700 1 -- 172.21.15.86:0/4118422160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974004880 con 0x55b9d6b27a60 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 -- 172.21.15.86:0/4118422160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 msgr2=0x55b9d6b2e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e9e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6974008fc0 tx=0x7f6974009920 comp rx=0 tx=0).stop 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 -- 172.21.15.86:0/4118422160 shutdown_connections 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b27a60 0x55b9d6b2e9e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b25e60 0x55b9d6b2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 --2- 172.21.15.86:0/4118422160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b270a0 0x55b9d6b37240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 -- 172.21.15.86:0/4118422160 >> 172.21.15.86:0/4118422160 conn(0x55b9d6a9b2a0 msgr2=0x55b9d6b2e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 -- 172.21.15.86:0/4118422160 shutdown_connections 2024-03-22T19:10:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.865+0000 7f6995545b80 1 -- 172.21.15.86:0/4118422160 wait complete. 2024-03-22T19:10:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.866+0000 7f6995545b80 1 -- start start 2024-03-22T19:10:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.866+0000 7f6995545b80 1 Processor -- start 2024-03-22T19:10:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.866+0000 7f6995545b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b36110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6995545b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b36650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6995545b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b8d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6995545b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6b36d30 con 0x55b9d6b270a0 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6995545b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6b8dba0 con 0x55b9d6b27a60 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6995545b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d6b8dce0 con 0x55b9d6b25e60 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6995545b80 10 client.0 _ll_register_callbacks cb 0x7f6968099210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b36650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b36650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43160/0 (socket says 172.21.15.86:43160) 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 -- 172.21.15.86:0/334970894 learned_addr learned my addr 172.21.15.86:0/334970894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697b400700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b8d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697aa00700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 -- 172.21.15.86:0/334970894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 msgr2=0x55b9d6b36110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b36110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 -- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 msgr2=0x55b9d6b8d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b8d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 -- 172.21.15.86:0/334970894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9d68f9430 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697b400700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b8d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f697a000700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b36650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6968005bf0 tx=0x7f6968097e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4336391a30ef0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.867+0000 7f6964c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6964c00700 1 -- 172.21.15.86:0/334970894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69680b7ea0 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6964c00700 1 -- 172.21.15.86:0/334970894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6968099d20 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f696c01e040 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6964c00700 1 -- 172.21.15.86:0/334970894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69680b2260 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 10 client.15357 fetch_fsmap learned FSMap version 27 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6974008d30 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6964c00700 1 -- 172.21.15.86:0/334970894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69680a5020 con 0x55b9d6b270a0 2024-03-22T19:10:52.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6964c00700 1 -- 172.21.15.86:0/334970894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f69680b90b0 con 0x55b9d6b270a0 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 10 client.15357 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 10 client.15357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 20 client.15357 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b8f420 0x55b9d6b91840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 4 client.15357 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.868+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b9d68c6d70 con 0x55b9d6b8f420 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.869+0000 7f697aa00700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b8f420 0x55b9d6b91840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.869+0000 7f697aa00700 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b8f420 0x55b9d6b91840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b46263a87efd78 server_cookie=5689c44c03470155 in_seq=0 out_seq=0 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.869+0000 7f6964c00700 10 client.15357 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.870+0000 7f6964c00700 1 -- 172.21.15.86:0/334970894 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b9d68c6d70 con 0x55b9d6b8f420 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.870+0000 7f6964c00700 10 client.15357 handle_command_reply: tid=0 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.870+0000 7f6995545b80 1 client.15357 shutdown 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 msgr2=0x55b9d6b36650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b36650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6968005bf0 tx=0x7f6968097e60 comp rx=0 tx=0).stop 2024-03-22T19:10:52.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 shutdown_connections 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b9d6b8f420 0x55b9d6b91840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9d6b27a60 0x55b9d6b8d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9d6b25e60 0x55b9d6b36110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 --2- 172.21.15.86:0/334970894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9d6b270a0 0x55b9d6b36650 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 >> 172.21.15.86:0/334970894 conn(0x55b9d6a9b2a0 msgr2=0x55b9d6b2e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6964c00700 0 client.15357 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 shutdown_connections 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 1 -- 172.21.15.86:0/334970894 wait complete. 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.871+0000 7f6995545b80 20 client.15357 trim_cache size 0 max 16384 2024-03-22T19:10:52.876 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:52.877 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:10:52.877 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "072ab60b-9243-4176-b7bb-d20c257fb3bc", 2024-03-22T19:10:52.877 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:10:52.877 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:52.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69600a0a90 msgr2=0x7f69600a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69600a0a90 0x7f69600a2f50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55b9d6b25c00 tx=0x7f697400b040 comp rx=0 tx=0).stop 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 msgr2=0x7f697c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 0x7f697c199200 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f696800e970 tx=0x7f696800ec80 comp rx=0 tx=0).stop 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 shutdown_connections 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f69600a0a90 0x7f69600a2f50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c108a50 0x7f697c1a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c073810 0x7f697c198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 --2- 172.21.15.86:0/3530913705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f697c074df0 0x7f697c199200 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 >> 172.21.15.86:0/3530913705 conn(0x7f697c0fc0e0 msgr2=0x7f697c104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 shutdown_connections 2024-03-22T19:10:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:52.872+0000 7f697be00700 1 -- 172.21.15.86:0/3530913705 wait complete. 2024-03-22T19:10:52.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:10:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 -- 172.21.15.86:0/3704725298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 msgr2=0x7f7a90109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 --2- 172.21.15.86:0/3704725298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a90109190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800b0d0 tx=0x7f7a8800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 -- 172.21.15.86:0/3704725298 shutdown_connections 2024-03-22T19:10:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 --2- 172.21.15.86:0/3704725298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a90109190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 --2- 172.21.15.86:0/3704725298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a90100d50 0x7f7a90101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 --2- 172.21.15.86:0/3704725298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a901023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.135+0000 7f7a95000700 1 -- 172.21.15.86:0/3704725298 >> 172.21.15.86:0/3704725298 conn(0x7f7a900fc2c0 msgr2=0x7f7a900fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.136+0000 7f7a95000700 1 -- 172.21.15.86:0/3704725298 shutdown_connections 2024-03-22T19:10:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.136+0000 7f7a95000700 1 -- 172.21.15.86:0/3704725298 wait complete. 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 Processor -- start 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 -- start start 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a90100d50 0x7f7a90198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a90199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a901a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a901a0b20 con 0x7f7a901028e0 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a901a0c60 con 0x7f7a90100d50 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a95000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a901a0df0 con 0x7f7a90101f40 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a901a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a8d400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a90199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a8d400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a90199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:35432/0 (socket says 172.21.15.86:35432) 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a8d400700 1 -- 172.21.15.86:0/750965020 learned_addr learned my addr 172.21.15.86:0/750965020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.137+0000 7f7a8de00700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a90100d50 0x7f7a90198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8e800700 1 -- 172.21.15.86:0/750965020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 msgr2=0x7f7a90199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8e800700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a90199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8e800700 1 -- 172.21.15.86:0/750965020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a90100d50 msgr2=0x7f7a90198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8e800700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a90100d50 0x7f7a90198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8e800700 1 -- 172.21.15.86:0/750965020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a88009d00 con 0x7f7a901028e0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8d400700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a90199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a8e800700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a901a0550 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a88007b60 tx=0x7f7a88008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a86a00700 1 -- 172.21.15.86:0/750965020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8800e040 con 0x7f7a901028e0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a86a00700 1 -- 172.21.15.86:0/750965020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a88005070 con 0x7f7a901028e0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a901a1070 con 0x7f7a901028e0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a86a00700 1 -- 172.21.15.86:0/750965020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8801d6d0 con 0x7f7a901028e0 2024-03-22T19:10:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.138+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a901a1440 con 0x7f7a901028e0 2024-03-22T19:10:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.141+0000 7f7a86a00700 1 -- 172.21.15.86:0/750965020 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f7a880051e0 con 0x7f7a901028e0 2024-03-22T19:10:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.142+0000 7f7aa8a9db80 1 Processor -- start 2024-03-22T19:10:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 -- start start 2024-03-22T19:10:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a86a00700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a740a0a90 0x7f7a740a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb129bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb12913990 con 0x55fb129bc420 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb12911350 con 0x55fb129bac20 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1290cc40 con 0x55fb129bb580 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a86a00700 1 -- 172.21.15.86:0/750965020 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7a880ccec0 con 0x7f7a901028e0 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a8d400700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a740a0a90 0x7f7a740a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129bb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53242/0 (socket says 172.21.15.86:53242) 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a8de00700 1 -- 172.21.15.86:0/1869157664 learned_addr learned my addr 172.21.15.86:0/1869157664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a8e800700 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb129bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.143+0000 7f7a8d400700 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8de00700 1 -- 172.21.15.86:0/1869157664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 msgr2=0x55fb129bbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8de00700 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129bbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8de00700 1 -- 172.21.15.86:0/1869157664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 msgr2=0x55fb129bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8de00700 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb129bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8de00700 1 -- 172.21.15.86:0/1869157664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb1278e3b0 con 0x55fb129bac20 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8e800700 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb129bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8d400700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a740a0a90 0x7f7a740a2f50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009fd0 tx=0x7f7a7c013d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a8de00700 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129bb040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009cb0 tx=0x7f7a80011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8854cfcccf126b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a7b400700 1 -- 172.21.15.86:0/1869157664 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80011d80 con 0x55fb129bac20 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a7b400700 1 -- 172.21.15.86:0/1869157664 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a800044e0 con 0x55fb129bac20 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.144+0000 7f7a7b400700 1 -- 172.21.15.86:0/1869157664 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80019430 con 0x55fb129bac20 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/1869157664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 msgr2=0x55fb129bb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129bb040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009cb0 tx=0x7f7a80011440 comp rx=0 tx=0).stop 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/1869157664 shutdown_connections 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb129bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129bb040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/1869157664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/1869157664 >> 172.21.15.86:0/1869157664 conn(0x55fb12930220 msgr2=0x55fb129b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/1869157664 shutdown_connections 2024-03-22T19:10:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.145+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/1869157664 wait complete. 2024-03-22T19:10:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.146+0000 7f7aa8a9db80 1 -- start start 2024-03-22T19:10:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.146+0000 7f7aa8a9db80 1 Processor -- start 2024-03-22T19:10:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.146+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bac20 0x55fb129c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.146+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.146+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 0x55fb129cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb128a8d80 con 0x55fb129bac20 2024-03-22T19:10:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb128a8a20 con 0x55fb129bc420 2024-03-22T19:10:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb127906a0 con 0x55fb129bb580 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7aa8a9db80 10 client.0 _ll_register_callbacks cb 0x55fb127143a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bac20 0x55fb129c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 0x55fb129cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 0x55fb129cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53258/0 (socket says 172.21.15.86:53258) 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bac20 0x55fb129c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43184/0 (socket says 172.21.15.86:43184) 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 -- 172.21.15.86:0/3020229015 learned_addr learned my addr 172.21.15.86:0/3020229015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8d400700 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 -- 172.21.15.86:0/3020229015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 msgr2=0x55fb129c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 -- 172.21.15.86:0/3020229015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bac20 msgr2=0x55fb129c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bac20 0x55fb129c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.147+0000 7f7a8e800700 1 -- 172.21.15.86:0/3020229015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a901a1440 con 0x55fb129bc420 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7a8e800700 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 0x55fb129cbfd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a880ca9b0 tx=0x7f7a8809a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc4fd6774a0fa65 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7a6fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:10:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7a6fe00700 1 -- 172.21.15.86:0/3020229015 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8809acd0 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb129adad0 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7a6fe00700 1 -- 172.21.15.86:0/3020229015 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7a8809a5d0 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7a6fe00700 1 -- 172.21.15.86:0/3020229015 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a8809e400 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7aa8a9db80 10 client.25394 fetch_fsmap learned FSMap version 27 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7a80009960 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.148+0000 7f7a6fe00700 1 -- 172.21.15.86:0/3020229015 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a88064020 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7a6fe00700 1 -- 172.21.15.86:0/3020229015 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a88024300 con 0x55fb129bc420 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7aa8a9db80 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7aa8a9db80 10 client.25394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7aa8a9db80 20 client.25394 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb129d0580 0x55fb129d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7aa8a9db80 4 client.25394 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb12905700 con 0x55fb129d0580 2024-03-22T19:10:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.149+0000 7f7a8de00700 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb129d0580 0x55fb129d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.150+0000 7f7a8de00700 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb129d0580 0x55fb129d2970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23bdec8f84f477f1 server_cookie=ffd14f238e0424f3 in_seq=0 out_seq=0 2024-03-22T19:10:53.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.150+0000 7f7a6fe00700 10 client.25394 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:53.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.150+0000 7f7a6fe00700 1 -- 172.21.15.86:0/3020229015 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fb12905700 con 0x55fb129d0580 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.150+0000 7f7a6fe00700 10 client.25394 handle_command_reply: tid=0 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.150+0000 7f7aa8a9db80 1 client.25394 shutdown 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 msgr2=0x55fb129cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 0x55fb129cbfd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a880ca9b0 tx=0x7f7a8809a420 comp rx=0 tx=0).stop 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 shutdown_connections 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb129d0580 0x55fb129d2970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bc420 0x55fb129cbfd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bac20 0x55fb129c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/3020229015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb129c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 >> 172.21.15.86:0/3020229015 conn(0x55fb12930220 msgr2=0x55fb129b51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7a6fe00700 0 client.25394 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 shutdown_connections 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/3020229015 wait complete. 2024-03-22T19:10:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.151+0000 7f7aa8a9db80 20 client.25394 trim_cache size 0 max 16384 2024-03-22T19:10:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 Processor -- start 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 -- start start 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bb580 0x55fb129cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 0x55fb129bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb129c4660 con 0x55fb129bb580 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb129d7660 con 0x55fb129bac20 2024-03-22T19:10:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.168+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb129b6930 con 0x55fb129bc420 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bb580 0x55fb129cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 0x55fb129bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 0x55fb129bc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:35464/0 (socket says 172.21.15.86:35464) 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53268/0 (socket says 172.21.15.86:53268) 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 -- 172.21.15.86:0/4111570006 learned_addr learned my addr 172.21.15.86:0/4111570006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 -- 172.21.15.86:0/4111570006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 msgr2=0x55fb129c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 -- 172.21.15.86:0/4111570006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bb580 msgr2=0x55fb129cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bb580 0x55fb129cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8e800700 1 -- 172.21.15.86:0/4111570006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb1273ab60 con 0x55fb129bc420 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.169+0000 7f7a8de00700 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7a8e800700 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 0x55fb129bc800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a88005a30 tx=0x7f7a8809aae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df692dafe6cbba87 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7a7be00700 1 -- 172.21.15.86:0/4111570006 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8809acd0 con 0x55fb129bc420 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7a7be00700 1 -- 172.21.15.86:0/4111570006 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a88064070 con 0x55fb129bc420 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7a7be00700 1 -- 172.21.15.86:0/4111570006 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88098970 con 0x55fb129bc420 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/4111570006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 msgr2=0x55fb129bc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 0x55fb129bc800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a88005a30 tx=0x7f7a8809aae0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.170+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/4111570006 shutdown_connections 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bc420 0x55fb129bc800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb129c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/4111570006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bb580 0x55fb129cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/4111570006 >> 172.21.15.86:0/4111570006 conn(0x55fb12930220 msgr2=0x55fb129c2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/4111570006 shutdown_connections 2024-03-22T19:10:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/4111570006 wait complete. 2024-03-22T19:10:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 -- start start 2024-03-22T19:10:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.171+0000 7f7aa8a9db80 1 Processor -- start 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb12a1c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb12a1e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb12a20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb12a214d0 con 0x55fb129bc420 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb12a21640 con 0x55fb129bac20 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb12a21800 con 0x55fb129bb580 2024-03-22T19:10:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7aa8a9db80 10 client.0 _ll_register_callbacks cb 0x7f7a88009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb12a1c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb12a20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb12a1c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:53270/0 (socket says 172.21.15.86:53270) 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 -- 172.21.15.86:0/45912230 learned_addr learned my addr 172.21.15.86:0/45912230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8d400700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb12a1e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 -- 172.21.15.86:0/45912230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 msgr2=0x55fb12a1e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb12a1e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 -- 172.21.15.86:0/45912230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 msgr2=0x55fb12a20d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb12a20d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8de00700 1 -- 172.21.15.86:0/45912230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb1278e3b0 con 0x55fb129bac20 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.172+0000 7f7a8d400700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb12a1e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7a8de00700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb12a1c1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a80011790 tx=0x7f7a80011840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47da459976b6fb2c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7a6fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:10:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7a6fe00700 1 -- 172.21.15.86:0/45912230 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80029050 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a800044e0 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7a6fe00700 1 -- 172.21.15.86:0/45912230 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7a8002a3b0 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7a6fe00700 1 -- 172.21.15.86:0/45912230 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a80005040 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7aa8a9db80 10 client.25400 fetch_fsmap learned FSMap version 27 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.173+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7a80009960 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7a6fe00700 1 -- 172.21.15.86:0/45912230 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a800044e0 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7a6fe00700 1 -- 172.21.15.86:0/45912230 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a80009ce0 con 0x55fb129bac20 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7aa8a9db80 10 client.25400 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7aa8a9db80 10 client.25400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7aa8a9db80 20 client.25400 populate_metadata read hostname 'smithi086' 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb12a23190 0x55fb12a255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7aa8a9db80 4 client.25400 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fb1275bcf0 con 0x55fb12a23190 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.174+0000 7f7a8d400700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb12a23190 0x55fb12a255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7a6fe00700 10 client.25400 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7a8d400700 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb12a23190 0x55fb12a255b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6954d058d94cc27 server_cookie=a1928e3b332800fa in_seq=0 out_seq=0 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7a6fe00700 1 -- 172.21.15.86:0/45912230 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fb1275bcf0 con 0x55fb12a23190 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7a6fe00700 10 client.25400 handle_command_reply: tid=0 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 client.25400 shutdown 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 msgr2=0x55fb12a1c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb12a1c1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a80011790 tx=0x7f7a80011840 comp rx=0 tx=0).stop 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 shutdown_connections 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fb12a23190 0x55fb12a255b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb129bc420 0x55fb12a20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb129bac20 0x55fb12a1c1f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7a6fe00700 0 client.25400 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 --2- 172.21.15.86:0/45912230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb129bb580 0x55fb12a1e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 >> 172.21.15.86:0/45912230 conn(0x55fb12930220 msgr2=0x55fb129c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 shutdown_connections 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.175+0000 7f7aa8a9db80 1 -- 172.21.15.86:0/45912230 wait complete. 2024-03-22T19:10:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7aa8a9db80 20 client.25400 trim_cache size 0 max 16384 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: "072ab60b-9243-4176-b7bb-d20c257fb3bc": { 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "072ab60b-9243-4176-b7bb-d20c257fb3bc", 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:10:53.181 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a740a0a90 msgr2=0x7f7a740a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a740a0a90 0x7f7a740a2f50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009fd0 tx=0x7f7a7c013d90 comp rx=0 tx=0).stop 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 msgr2=0x7f7a901a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a901a0550 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a88007b60 tx=0x7f7a88008e40 comp rx=0 tx=0).stop 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 shutdown_connections 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a740a0a90 0x7f7a740a2f50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a901028e0 0x7f7a901a0550 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a90100d50 0x7f7a90198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 --2- 172.21.15.86:0/750965020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90101f40 0x7f7a90199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:10:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 >> 172.21.15.86:0/750965020 conn(0x7f7a900fc2c0 msgr2=0x7f7a90072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:10:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 shutdown_connections 2024-03-22T19:10:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:10:53.176+0000 7f7a95000700 1 -- 172.21.15.86:0/750965020 wait complete. 2024-03-22T19:10:53.196 INFO:tasks.cephfs.filesystem:scrub status for tag:072ab60b-9243-4176-b7bb-d20c257fb3bc - {'path': '/', 'tag': '072ab60b-9243-4176-b7bb-d20c257fb3bc', 'options': 'recursive,force'} 2024-03-22T19:10:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:53 smithi086 ceph-mon[38032]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-22T19:10:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:53 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3815519791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:53 smithi167 ceph-mon[47728]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-22T19:10:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:53 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3815519791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:53 smithi131 ceph-mon[78559]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-22T19:10:53.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:53 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3815519791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:10:54.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:54 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:10:54.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:54 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:10:54.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:54 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:10:54.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:54 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:10:54.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:54 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:10:54.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:54 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:10:54.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:54 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:10:54.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:54 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:10:54.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:54 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:10:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:54 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:10:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:54 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:10:54.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:54 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:10:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:55 smithi167 ceph-mon[47728]: pgmap v284: 97 pgs: 97 active+clean; 646 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-22T19:10:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:55 smithi131 ceph-mon[78559]: pgmap v284: 97 pgs: 97 active+clean; 646 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-22T19:10:55.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:55 smithi086 ceph-mon[38032]: pgmap v284: 97 pgs: 97 active+clean; 646 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-22T19:10:57.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:57 smithi131 ceph-mon[78559]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-03-22T19:10:57.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:57 smithi086 ceph-mon[38032]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-03-22T19:10:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:57 smithi167 ceph-mon[47728]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-03-22T19:10:58.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:58 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:10:58.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:58 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:10:58.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:58 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:10:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:10:59 smithi131 ceph-mon[78559]: pgmap v286: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-22T19:10:59.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:10:59 smithi086 ceph-mon[38032]: pgmap v286: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-22T19:10:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:10:59 smithi167 ceph-mon[47728]: pgmap v286: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-22T19:11:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:01 smithi086 ceph-mon[38032]: pgmap v287: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-22T19:11:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:01 smithi167 ceph-mon[47728]: pgmap v287: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-22T19:11:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:01 smithi131 ceph-mon[78559]: pgmap v287: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-22T19:11:03.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:03 smithi086 ceph-mon[38032]: pgmap v288: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:11:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:03 smithi167 ceph-mon[47728]: pgmap v288: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:11:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:03 smithi131 ceph-mon[78559]: pgmap v288: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:11:05.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:05 smithi086 ceph-mon[38032]: pgmap v289: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:11:05.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:05 smithi167 ceph-mon[47728]: pgmap v289: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:11:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:05 smithi131 ceph-mon[78559]: pgmap v289: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-22T19:11:07.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:07 smithi131 ceph-mon[78559]: pgmap v290: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-22T19:11:07.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:07 smithi086 ceph-mon[38032]: pgmap v290: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-22T19:11:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:07 smithi167 ceph-mon[47728]: pgmap v290: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-22T19:11:09.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:09 smithi086 ceph-mon[38032]: pgmap v291: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-22T19:11:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:09 smithi167 ceph-mon[47728]: pgmap v291: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-22T19:11:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:09 smithi131 ceph-mon[78559]: pgmap v291: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-22T19:11:11.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:11 smithi086 ceph-mon[38032]: pgmap v292: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:11:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:11 smithi167 ceph-mon[47728]: pgmap v292: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:11:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:11 smithi131 ceph-mon[78559]: pgmap v292: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:11:13.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:13 smithi086 ceph-mon[38032]: pgmap v293: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-22T19:11:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:13 smithi167 ceph-mon[47728]: pgmap v293: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-22T19:11:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:13 smithi131 ceph-mon[78559]: pgmap v293: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-22T19:11:15.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:15 smithi086 ceph-mon[38032]: pgmap v294: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:15.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:15 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:11:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:15 smithi167 ceph-mon[47728]: pgmap v294: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:15 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:11:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:15 smithi131 ceph-mon[78559]: pgmap v294: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:15 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: pgmap v295: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:17 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: pgmap v295: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:17 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: pgmap v295: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:17.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:17.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:17 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:11:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:11:18.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:11:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:11:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:11:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:11:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:11:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:19 smithi131 ceph-mon[78559]: pgmap v296: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:19.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:19 smithi086 ceph-mon[38032]: pgmap v296: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:19 smithi167 ceph-mon[47728]: pgmap v296: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:21.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:21 smithi086 ceph-mon[38032]: pgmap v297: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:21 smithi167 ceph-mon[47728]: pgmap v297: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:21 smithi131 ceph-mon[78559]: pgmap v297: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:23.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 -- 172.21.15.86:0/17684455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c100d40 msgr2=0x7f986c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 --2- 172.21.15.86:0/17684455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c100d40 0x7f986c101160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f985c0099c0 tx=0x7f985c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 -- 172.21.15.86:0/17684455 shutdown_connections 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 --2- 172.21.15.86:0/17684455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1028d0 0x7f986c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 --2- 172.21.15.86:0/17684455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c100d40 0x7f986c101160 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 --2- 172.21.15.86:0/17684455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 0x7f986c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 -- 172.21.15.86:0/17684455 >> 172.21.15.86:0/17684455 conn(0x7f986c0fc2b0 msgr2=0x7f986c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 -- 172.21.15.86:0/17684455 shutdown_connections 2024-03-22T19:11:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.403+0000 7f9871200700 1 -- 172.21.15.86:0/17684455 wait complete. 2024-03-22T19:11:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 Processor -- start 2024-03-22T19:11:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 -- start start 2024-03-22T19:11:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c100d40 0x7f986c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 0x7f986c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 0x7f986c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1a0a20 con 0x7f986c1028d0 2024-03-22T19:11:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1a0b60 con 0x7f986c101f30 2024-03-22T19:11:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.405+0000 7f9871200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1a0cf0 con 0x7f986c100d40 2024-03-22T19:11:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 0x7f986c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f9869400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 0x7f986c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 0x7f986c1a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38804/0 (socket says 172.21.15.86:38804) 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 -- 172.21.15.86:0/3211023602 learned_addr learned my addr 172.21.15.86:0/3211023602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f9869e00700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c100d40 0x7f986c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 -- 172.21.15.86:0/3211023602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c100d40 msgr2=0x7f986c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c100d40 0x7f986c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 -- 172.21.15.86:0/3211023602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 msgr2=0x7f986c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 0x7f986c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 -- 172.21.15.86:0/3211023602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0095f0 con 0x7f986c1028d0 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f9869400700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 0x7f986c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f986a800700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 0x7f986c1a0450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f986400ef30 tx=0x7f986400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f9862a00700 1 -- 172.21.15.86:0/3211023602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864018070 con 0x7f986c1028d0 2024-03-22T19:11:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f9862a00700 1 -- 172.21.15.86:0/3211023602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f986400f040 con 0x7f986c1028d0 2024-03-22T19:11:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.406+0000 7f9862a00700 1 -- 172.21.15.86:0/3211023602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400cd00 con 0x7f986c1028d0 2024-03-22T19:11:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.407+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c1a0f70 con 0x7f986c1028d0 2024-03-22T19:11:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.407+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c1a13a0 con 0x7f986c1028d0 2024-03-22T19:11:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.410+0000 7f9862a00700 1 -- 172.21.15.86:0/3211023602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9864003a00 con 0x7f986c1028d0 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 Processor -- start 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 -- start start 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9862a00700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98500a0b10 0x7f98500a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edae6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edae6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edae78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608eda3e990 con 0x5608edae5c20 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608eda3c350 con 0x5608edae7420 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608eda37c40 con 0x5608edae6580 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.411+0000 7f9862a00700 1 -- 172.21.15.86:0/3211023602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f98640cba20 con 0x7f986c1028d0 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edae6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edae6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38808/0 (socket says 172.21.15.86:38808) 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 -- 172.21.15.86:0/3286139919 learned_addr learned my addr 172.21.15.86:0/3286139919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f986a800700 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edae78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869400700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98500a0b10 0x7f98500a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869400700 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edae6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 -- 172.21.15.86:0/3286139919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 msgr2=0x5608edae6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edae6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 -- 172.21.15.86:0/3286139919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 msgr2=0x5608edae78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edae78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 -- 172.21.15.86:0/3286139919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ed8b93b0 con 0x5608edae5c20 2024-03-22T19:11:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.412+0000 7f9869e00700 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edae6040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f985c006010 tx=0x7f985c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e91fec20b95658 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.413+0000 7f9869400700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98500a0b10 0x7f98500a2fd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f98580097c0 tx=0x7f9858009750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.413+0000 7f9857400700 1 -- 172.21.15.86:0/3286139919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00ba10 con 0x5608edae5c20 2024-03-22T19:11:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.413+0000 7f9857400700 1 -- 172.21.15.86:0/3286139919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f985c01e040 con 0x5608edae5c20 2024-03-22T19:11:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.413+0000 7f9857400700 1 -- 172.21.15.86:0/3286139919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c003b30 con 0x5608edae5c20 2024-03-22T19:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.413+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3286139919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 msgr2=0x5608edae6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.413+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edae6040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f985c006010 tx=0x7f985c005bc0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3286139919 shutdown_connections 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edae78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edae6040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3286139919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edae6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3286139919 >> 172.21.15.86:0/3286139919 conn(0x5608eda5b220 msgr2=0x5608edae0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3286139919 shutdown_connections 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3286139919 wait complete. 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 -- start start 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.414+0000 7f9884dc3b80 1 Processor -- start 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae5c20 0x5608edaefa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaeff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 0x5608edaf6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ed9d3d80 con 0x5608edae7420 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ed9d3a20 con 0x5608edae6580 2024-03-22T19:11:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ed8bb6a0 con 0x5608edae5c20 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9884dc3b80 10 client.0 _ll_register_callbacks cb 0x5608ed83f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 0x5608edaf6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9869e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae5c20 0x5608edaefa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9869400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaeff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 0x5608edaf6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38814/0 (socket says 172.21.15.86:38814) 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 -- 172.21.15.86:0/3689166831 learned_addr learned my addr 172.21.15.86:0/3689166831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 -- 172.21.15.86:0/3689166831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae5c20 msgr2=0x5608edaefa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae5c20 0x5608edaefa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 -- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 msgr2=0x5608edaeff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaeff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f986a800700 1 -- 172.21.15.86:0/3689166831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c109ad0 con 0x5608edae7420 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.415+0000 7f9869400700 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaeff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f986a800700 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 0x5608edaf6fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f98640c95b0 tx=0x7f9864048c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11b85e7677d4057 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f984be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f984be00700 1 -- 172.21.15.86:0/3689166831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986401b320 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f984be00700 1 -- 172.21.15.86:0/3689166831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f98640998d0 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608edad8ad0 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f984be00700 1 -- 172.21.15.86:0/3689166831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f986400ce60 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f9884dc3b80 10 client.15387 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608ed865b60 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f984be00700 1 -- 172.21.15.86:0/3689166831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98640b5020 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f984be00700 1 -- 172.21.15.86:0/3689166831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f986409f0d0 con 0x5608edae7420 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f9884dc3b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.416+0000 7f9884dc3b80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f9884dc3b80 20 client.15387 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edafb860 0x5608edafdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f9884dc3b80 4 client.15387 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608eda30700 con 0x5608edafb860 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f9869e00700 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edafb860 0x5608edafdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f9869e00700 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edafb860 0x5608edafdc50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a78120597efab3 server_cookie=1c3ee24d57fb7187 in_seq=0 out_seq=0 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.417+0000 7f984be00700 10 client.15387 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.418+0000 7f984be00700 1 -- 172.21.15.86:0/3689166831 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5608eda30700 con 0x5608edafb860 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.418+0000 7f984be00700 10 client.15387 handle_command_reply: tid=0 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.418+0000 7f9884dc3b80 1 client.15387 shutdown 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.418+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 msgr2=0x5608edaf6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.418+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 0x5608edaf6fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f98640c95b0 tx=0x7f9864048c60 comp rx=0 tx=0).stop 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 shutdown_connections 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edafb860 0x5608edafdc50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae7420 0x5608edaf6fb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae5c20 0x5608edaefa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f984be00700 0 client.15387 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/3689166831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaeff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 >> 172.21.15.86:0/3689166831 conn(0x5608eda5b220 msgr2=0x5608edae01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 shutdown_connections 2024-03-22T19:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 1 -- 172.21.15.86:0/3689166831 wait complete. 2024-03-22T19:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.419+0000 7f9884dc3b80 20 client.15387 trim_cache size 0 max 16384 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 Processor -- start 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 -- start start 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edaf4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaf7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae7420 0x5608edaeef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608edaef5a0 con 0x5608edae5c20 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608edafb660 con 0x5608edae6580 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608edae63c0 con 0x5608edae7420 2024-03-22T19:11:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9869e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edaf4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.436+0000 7f9869e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edaf4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38822/0 (socket says 172.21.15.86:38822) 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 -- 172.21.15.86:0/1363447822 learned_addr learned my addr 172.21.15.86:0/1363447822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869400700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaf7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f986a800700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae7420 0x5608edaeef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 -- 172.21.15.86:0/1363447822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae7420 msgr2=0x5608edaeef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae7420 0x5608edaeef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 -- 172.21.15.86:0/1363447822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 msgr2=0x5608edaf7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaf7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 -- 172.21.15.86:0/1363447822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ed9fdea0 con 0x5608edae5c20 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f986a800700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae7420 0x5608edaeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869400700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaf7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9869e00700 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edaf4f60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5608edae25a0 tx=0x7f985c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928f5949b8230f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9857e00700 1 -- 172.21.15.86:0/1363447822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c031400 con 0x5608edae5c20 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9857e00700 1 -- 172.21.15.86:0/1363447822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f985c031e80 con 0x5608edae5c20 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.437+0000 7f9857e00700 1 -- 172.21.15.86:0/1363447822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c0037d0 con 0x5608edae5c20 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 -- 172.21.15.86:0/1363447822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 msgr2=0x5608edaf4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edaf4f60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5608edae25a0 tx=0x7f985c01a040 comp rx=0 tx=0).stop 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 -- 172.21.15.86:0/1363447822 shutdown_connections 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae7420 0x5608edaeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edaf4f60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/1363447822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae6580 0x5608edaf7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 -- 172.21.15.86:0/1363447822 >> 172.21.15.86:0/1363447822 conn(0x5608eda5b220 msgr2=0x5608edaed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 -- 172.21.15.86:0/1363447822 shutdown_connections 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 -- 172.21.15.86:0/1363447822 wait complete. 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.438+0000 7f9884dc3b80 1 -- start start 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 Processor -- start 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edb47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edb49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edb4bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608edb4c310 con 0x5608edae5c20 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608edb4c450 con 0x5608edae7420 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608edb4c590 con 0x5608edae6580 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.439+0000 7f9884dc3b80 10 client.0 _ll_register_callbacks cb 0x7f9864098d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edb47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edb47330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38828/0 (socket says 172.21.15.86:38828) 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 -- 172.21.15.86:0/2705725898 learned_addr learned my addr 172.21.15.86:0/2705725898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f986a800700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edb4bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869400700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edb49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 -- 172.21.15.86:0/2705725898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 msgr2=0x5608edb49880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edb49880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 -- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 msgr2=0x5608edb4bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edb4bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 -- 172.21.15.86:0/2705725898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608ed8b93b0 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f986a800700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edb4bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f9869e00700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edb47330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f985c0058e0 tx=0x7f985c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f1f849f601f647 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.440+0000 7f984be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f984be00700 1 -- 172.21.15.86:0/2705725898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c005720 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f984be00700 1 -- 172.21.15.86:0/2705725898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f985c005130 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608ed886cf0 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f984be00700 1 -- 172.21.15.86:0/2705725898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f985c031400 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 10 client.15393 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608ed8bdcd0 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f984be00700 1 -- 172.21.15.86:0/2705725898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608ed886cf0 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f984be00700 1 -- 172.21.15.86:0/2705725898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f985c038070 con 0x5608edae5c20 2024-03-22T19:11:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 20 client.15393 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edb4d9e0 0x5608edb4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 4 client.15393 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.441+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608ed8bbce0 con 0x5608edb4d9e0 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.442+0000 7f9869400700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edb4d9e0 0x5608edb4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.442+0000 7f984be00700 10 client.15393 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.442+0000 7f9869400700 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edb4d9e0 0x5608edb4fdd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e1886633fe4589 server_cookie=cc73edddec3ebe61 in_seq=0 out_seq=0 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.442+0000 7f984be00700 1 -- 172.21.15.86:0/2705725898 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5608ed8bbce0 con 0x5608edb4d9e0 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.442+0000 7f984be00700 10 client.15393 handle_command_reply: tid=0 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.442+0000 7f9884dc3b80 1 client.15393 shutdown 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 msgr2=0x5608edb47330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edb47330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f985c0058e0 tx=0x7f985c009310 comp rx=0 tx=0).stop 2024-03-22T19:11:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 shutdown_connections 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5608edb4d9e0 0x5608edb4fdd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608edae7420 0x5608edb4bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608edae5c20 0x5608edb47330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f984be00700 0 client.15393 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 --2- 172.21.15.86:0/2705725898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608edae6580 0x5608edb49880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 >> 172.21.15.86:0/2705725898 conn(0x5608eda5b220 msgr2=0x5608edaeef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 shutdown_connections 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 1 -- 172.21.15.86:0/2705725898 wait complete. 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9884dc3b80 20 client.15393 trim_cache size 0 max 16384 2024-03-22T19:11:23.451 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:11:23.452 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:11:23.452 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:11:23.452 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:11:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98500a0b10 msgr2=0x7f98500a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9871200700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98500a0b10 0x7f98500a2fd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f98580097c0 tx=0x7f9858009750 comp rx=0 tx=0).stop 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 msgr2=0x7f986c1a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.443+0000 7f9871200700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 0x7f986c1a0450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f986400ef30 tx=0x7f986400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 shutdown_connections 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98500a0b10 0x7f98500a2fd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c1028d0 0x7f986c1a0450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c100d40 0x7f986c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 --2- 172.21.15.86:0/3211023602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c101f30 0x7f986c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 >> 172.21.15.86:0/3211023602 conn(0x7f986c0fc2b0 msgr2=0x7f986c107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 shutdown_connections 2024-03-22T19:11:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.444+0000 7f9871200700 1 -- 172.21.15.86:0/3211023602 wait complete. 2024-03-22T19:11:23.464 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:11:23.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:11:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:23 smithi086 ceph-mon[38032]: pgmap v298: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:23 smithi167 ceph-mon[47728]: pgmap v298: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 -- 172.21.15.86:0/917798224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 msgr2=0x7f1fe8101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 --2- 172.21.15.86:0/917798224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8101070 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80099c0 tx=0x7f1fd8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 -- 172.21.15.86:0/917798224 shutdown_connections 2024-03-22T19:11:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 --2- 172.21.15.86:0/917798224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe81027e0 0x7f1fe8109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 --2- 172.21.15.86:0/917798224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8101070 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 --2- 172.21.15.86:0/917798224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8101e40 0x7f1fe81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.654+0000 7f1fee600700 1 -- 172.21.15.86:0/917798224 >> 172.21.15.86:0/917798224 conn(0x7f1fe80fc1c0 msgr2=0x7f1fe80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.655+0000 7f1fee600700 1 -- 172.21.15.86:0/917798224 shutdown_connections 2024-03-22T19:11:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.655+0000 7f1fee600700 1 -- 172.21.15.86:0/917798224 wait complete. 2024-03-22T19:11:23.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 Processor -- start 2024-03-22T19:11:23.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 -- start start 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8101e40 0x7f1fe81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe81027e0 0x7f1fe81a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81a0960 con 0x7f1fe8100c50 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81a0aa0 con 0x7f1fe8101e40 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fee600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81a0c30 con 0x7f1fe81027e0 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fe7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fe7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38838/0 (socket says 172.21.15.86:38838) 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fe7400700 1 -- 172.21.15.86:0/476563881 learned_addr learned my addr 172.21.15.86:0/476563881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fe6a00700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8101e40 0x7f1fe81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.656+0000 7f1fe7e00700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe81027e0 0x7f1fe81a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fe7400700 1 -- 172.21.15.86:0/476563881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe81027e0 msgr2=0x7f1fe81a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fe7400700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe81027e0 0x7f1fe81a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fe7400700 1 -- 172.21.15.86:0/476563881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8101e40 msgr2=0x7f1fe81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fe7400700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8101e40 0x7f1fe81993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fe7400700 1 -- 172.21.15.86:0/476563881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd4009520 con 0x7f1fe8100c50 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fe7400700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8198e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8006010 tx=0x7f1fd800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd801d070 con 0x7f1fe8100c50 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fd8005020 con 0x7f1fe8100c50 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd80095f0 con 0x7f1fe8100c50 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8004b10 con 0x7f1fe8100c50 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.657+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe81a1140 con 0x7f1fe8100c50 2024-03-22T19:11:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.658+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe80679e0 con 0x7f1fe8100c50 2024-03-22T19:11:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.660+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f1fd8005190 con 0x7f1fe8100c50 2024-03-22T19:11:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.661+0000 7f1fdfe00700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1fcc0a0b10 0x7f1fcc0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.662+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1fd80cc9b0 con 0x7f1fe8100c50 2024-03-22T19:11:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.662+0000 7f1fe6a00700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1fcc0a0b10 0x7f1fcc0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.662+0000 7f1fe6a00700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1fcc0a0b10 0x7f1fcc0a2fd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009df0 tx=0x7f1fd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.666+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1fd8096400 con 0x7f1fe8100c50 2024-03-22T19:11:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:23 smithi131 ceph-mon[78559]: pgmap v298: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.921+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fe81a1a20 con 0x7f1fe8100c50 2024-03-22T19:11:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.922+0000 7f1fdfe00700 1 -- 172.21.15.86:0/476563881 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f1fd8099a20 con 0x7f1fe8100c50 2024-03-22T19:11:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:11:23.923 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:10:44.254849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1fcc0a0b10 msgr2=0x7f1fcc0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1fcc0a0b10 0x7f1fcc0a2fd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009df0 tx=0x7f1fd4009260 comp rx=0 tx=0).stop 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 msgr2=0x7f1fe8198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8198e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8006010 tx=0x7f1fd800f690 comp rx=0 tx=0).stop 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 shutdown_connections 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1fcc0a0b10 0x7f1fcc0a2fd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe81027e0 0x7f1fe81a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe8100c50 0x7f1fe8198e80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 --2- 172.21.15.86:0/476563881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8101e40 0x7f1fe81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 >> 172.21.15.86:0/476563881 conn(0x7f1fe80fc1c0 msgr2=0x7f1fe8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.924+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 shutdown_connections 2024-03-22T19:11:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:23.925+0000 7f1fee600700 1 -- 172.21.15.86:0/476563881 wait complete. 2024-03-22T19:11:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-22T19:11:23.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:11:24.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 -- 172.21.15.86:0/1949760531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec1028e0 msgr2=0x7efeec109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 --2- 172.21.15.86:0/1949760531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec1028e0 0x7efeec109190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efee400b0d0 tx=0x7efee400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 -- 172.21.15.86:0/1949760531 shutdown_connections 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 --2- 172.21.15.86:0/1949760531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec1028e0 0x7efeec109190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 --2- 172.21.15.86:0/1949760531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec100d50 0x7efeec101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 --2- 172.21.15.86:0/1949760531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efeec101f40 0x7efeec1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 -- 172.21.15.86:0/1949760531 >> 172.21.15.86:0/1949760531 conn(0x7efeec0fc2c0 msgr2=0x7efeec0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 -- 172.21.15.86:0/1949760531 shutdown_connections 2024-03-22T19:11:24.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.178+0000 7efef2400700 1 -- 172.21.15.86:0/1949760531 wait complete. 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 Processor -- start 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 -- start start 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 0x7efeec198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efeec101f40 0x7efeec199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec1028e0 0x7efeec1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1a0b20 con 0x7efeec100d50 2024-03-22T19:11:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1a0c60 con 0x7efeec1028e0 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efef2400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1a0df0 con 0x7efeec101f40 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.179+0000 7efeeb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 0x7efeec198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 0x7efeec198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38850/0 (socket says 172.21.15.86:38850) 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 -- 172.21.15.86:0/4070199293 learned_addr learned my addr 172.21.15.86:0/4070199293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeaa00700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efeec101f40 0x7efeec199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeebe00700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec1028e0 0x7efeec1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 -- 172.21.15.86:0/4070199293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efeec101f40 msgr2=0x7efeec199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efeec101f40 0x7efeec199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 -- 172.21.15.86:0/4070199293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec1028e0 msgr2=0x7efeec1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec1028e0 0x7efeec1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 -- 172.21.15.86:0/4070199293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4009d00 con 0x7efeec100d50 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efeeb400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 0x7efeec198ff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e970 tx=0x7efedc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efee3e00700 1 -- 172.21.15.86:0/4070199293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc009790 con 0x7efeec100d50 2024-03-22T19:11:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efee3e00700 1 -- 172.21.15.86:0/4070199293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efedc00c7e0 con 0x7efeec100d50 2024-03-22T19:11:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efee3e00700 1 -- 172.21.15.86:0/4070199293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc003c70 con 0x7efeec100d50 2024-03-22T19:11:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.180+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec1a10d0 con 0x7efeec100d50 2024-03-22T19:11:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.181+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec1a1590 con 0x7efeec100d50 2024-03-22T19:11:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.183+0000 7efee3e00700 1 -- 172.21.15.86:0/4070199293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7efedc00cc50 con 0x7efeec100d50 2024-03-22T19:11:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.185+0000 7eff05e89b80 1 Processor -- start 2024-03-22T19:11:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.185+0000 7eff05e89b80 1 -- start start 2024-03-22T19:11:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc038e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc039420 0x5565fc0398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7eff05e89b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fbf90990 con 0x5565fc039420 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efee3e00700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efed00a0a90 0x7efed00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7eff05e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fbf8e350 con 0x5565fc037c20 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7eff05e89b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fbf89c40 con 0x5565fc038580 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efee3e00700 1 -- 172.21.15.86:0/4070199293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7efedc014070 con 0x7efeec100d50 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeebe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc039420 0x5565fc0398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeebe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39956/0 (socket says 172.21.15.86:39956) 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeebe00700 1 -- 172.21.15.86:0/2133144940 learned_addr learned my addr 172.21.15.86:0/2133144940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeebe00700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efed00a0a90 0x7efed00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeb400700 1 -- 172.21.15.86:0/2133144940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 msgr2=0x5565fc038040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeb400700 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeb400700 1 -- 172.21.15.86:0/2133144940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc039420 msgr2=0x5565fc0398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeb400700 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc039420 0x5565fc0398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.186+0000 7efeeb400700 1 -- 172.21.15.86:0/2133144940 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565fbe0b3b0 con 0x5565fc038580 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.187+0000 7efeebe00700 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.187+0000 7efeeb400700 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc038e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efedc005950 tx=0x7efedc099790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2ad1b1d7b0199e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.187+0000 7efeebe00700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efed00a0a90 0x7efed00a2f50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efee400b0d0 tx=0x7efee400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.187+0000 7efee0c00700 1 -- 172.21.15.86:0/2133144940 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0b7ea0 con 0x5565fc038580 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.187+0000 7efee0c00700 1 -- 172.21.15.86:0/2133144940 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efedc0998b0 con 0x5565fc038580 2024-03-22T19:11:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.187+0000 7efee0c00700 1 -- 172.21.15.86:0/2133144940 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc099b10 con 0x5565fc038580 2024-03-22T19:11:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 -- 172.21.15.86:0/2133144940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 msgr2=0x5565fc038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc038e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efedc005950 tx=0x7efedc099790 comp rx=0 tx=0).stop 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 -- 172.21.15.86:0/2133144940 shutdown_connections 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc039420 0x5565fc0398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2133144940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc038e40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 -- 172.21.15.86:0/2133144940 >> 172.21.15.86:0/2133144940 conn(0x5565fbfad220 msgr2=0x5565fc032bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 -- 172.21.15.86:0/2133144940 shutdown_connections 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 -- 172.21.15.86:0/2133144940 wait complete. 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.188+0000 7eff05e89b80 1 -- start start 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 Processor -- start 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 0x5565fc041af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc042030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc039420 0x5565fc049070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fbf25d80 con 0x5565fc037c20 2024-03-22T19:11:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fbf25a20 con 0x5565fc039420 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fbe0d6a0 con 0x5565fc038580 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7eff05e89b80 10 client.0 _ll_register_callbacks cb 0x5565fbd913a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7efeeb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 0x5565fc041af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7efeeb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 0x5565fc041af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38872/0 (socket says 172.21.15.86:38872) 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7efeeaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7efeeb400700 1 -- 172.21.15.86:0/2456233589 learned_addr learned my addr 172.21.15.86:0/2456233589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.189+0000 7efeebe00700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc039420 0x5565fc049070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeb400700 1 -- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 msgr2=0x5565fc042030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeb400700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc042030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeb400700 1 -- 172.21.15.86:0/2456233589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc039420 msgr2=0x5565fc049070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeb400700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc039420 0x5565fc049070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeb400700 1 -- 172.21.15.86:0/2456233589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec109ae0 con 0x5565fc037c20 2024-03-22T19:11:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeaa00700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc042030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efeeb400700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 0x5565fc041af0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efedc0a1800 tx=0x7efedc005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea577bca81bcbba server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efed5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efed5600700 1 -- 172.21.15.86:0/2456233589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0b7ea0 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efed5600700 1 -- 172.21.15.86:0/2456233589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7efedc09d6e0 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565fc02aad0 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7efed5600700 1 -- 172.21.15.86:0/2456233589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efedc0b2230 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.190+0000 7eff05e89b80 10 client.15411 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7efeec1a1590 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7efed5600700 1 -- 172.21.15.86:0/2456233589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565fc02aad0 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7efed5600700 1 -- 172.21.15.86:0/2456233589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efedc0a19c0 con 0x5565fc037c20 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 10 client.15411 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 10 client.15411 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 20 client.15411 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc04d310 0x5565fc04f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 4 client.15411 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565fbf82700 con 0x5565fc04d310 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.191+0000 7efeeaa00700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc04d310 0x5565fc04f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.192+0000 7efed5600700 10 client.15411 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.192+0000 7efeeaa00700 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc04d310 0x5565fc04f700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7a18f6fc11c8fed server_cookie=ccf219afa1239df1 in_seq=0 out_seq=0 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7efed5600700 1 -- 172.21.15.86:0/2456233589 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5565fbf82700 con 0x5565fc04d310 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7efed5600700 10 client.15411 handle_command_reply: tid=0 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 client.15411 shutdown 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 msgr2=0x5565fc041af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 0x5565fc041af0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efedc0a1800 tx=0x7efedc005b40 comp rx=0 tx=0).stop 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 shutdown_connections 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc039420 0x5565fc049070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc037c20 0x5565fc041af0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc04d310 0x5565fc04f700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2456233589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc038580 0x5565fc042030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 >> 172.21.15.86:0/2456233589 conn(0x5565fbfad220 msgr2=0x5565fc0321f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7efed5600700 0 client.15411 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 shutdown_connections 2024-03-22T19:11:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 1 -- 172.21.15.86:0/2456233589 wait complete. 2024-03-22T19:11:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.193+0000 7eff05e89b80 20 client.15411 trim_cache size 0 max 16384 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.206+0000 7eff05e89b80 1 Processor -- start 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.206+0000 7eff05e89b80 1 -- start start 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc049030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc039800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7eff05e89b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fc038960 con 0x5565fc038580 2024-03-22T19:11:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7eff05e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fc02cf80 con 0x5565fc037c20 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7eff05e89b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fc03f5c0 con 0x5565fc039420 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc049030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc049030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38878/0 (socket says 172.21.15.86:38878) 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 -- 172.21.15.86:0/2294646392 learned_addr learned my addr 172.21.15.86:0/2294646392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeb400700 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeebe00700 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc039800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 -- 172.21.15.86:0/2294646392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 msgr2=0x5565fc039800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc039800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 -- 172.21.15.86:0/2294646392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 msgr2=0x5565fc038000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.207+0000 7efeeaa00700 1 -- 172.21.15.86:0/2294646392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565fbdb7b60 con 0x5565fc038580 2024-03-22T19:11:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7efeeaa00700 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc049030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5565fc050060 tx=0x7efed8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d55a8c25026e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7efee1600700 1 -- 172.21.15.86:0/2294646392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8017070 con 0x5565fc038580 2024-03-22T19:11:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7efee1600700 1 -- 172.21.15.86:0/2294646392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed8018040 con 0x5565fc038580 2024-03-22T19:11:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7efee1600700 1 -- 172.21.15.86:0/2294646392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8005440 con 0x5565fc038580 2024-03-22T19:11:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 -- 172.21.15.86:0/2294646392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 msgr2=0x5565fc049030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc049030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5565fc050060 tx=0x7efed8009260 comp rx=0 tx=0).stop 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 -- 172.21.15.86:0/2294646392 shutdown_connections 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc039800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc038000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 --2- 172.21.15.86:0/2294646392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc049030 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 -- 172.21.15.86:0/2294646392 >> 172.21.15.86:0/2294646392 conn(0x5565fbfad220 msgr2=0x5565fc041000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 -- 172.21.15.86:0/2294646392 shutdown_connections 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.208+0000 7eff05e89b80 1 -- 172.21.15.86:0/2294646392 wait complete. 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 -- start start 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 Processor -- start 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc044cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc045210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc048190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fc0486d0 con 0x5565fc038580 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fc048840 con 0x5565fc037c20 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.209+0000 7eff05e89b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fc048a00 con 0x5565fc039420 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7eff05e89b80 10 client.0 _ll_register_callbacks cb 0x7efed8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc045210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc044cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc045210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38884/0 (socket says 172.21.15.86:38884) 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc044cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39984/0 (socket says 172.21.15.86:39984) 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeaa00700 1 -- 172.21.15.86:0/1706771243 learned_addr learned my addr 172.21.15.86:0/1706771243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeebe00700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc048190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 -- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 msgr2=0x5565fc048190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc048190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 -- 172.21.15.86:0/1706771243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 msgr2=0x5565fc045210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc045210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 -- 172.21.15.86:0/1706771243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed8012170 con 0x5565fc037c20 2024-03-22T19:11:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeaa00700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc045210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efeeb400700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc044cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efedc000c00 tx=0x7efedc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e1e7eeeefd61a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efed5600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.210+0000 7efed5600700 1 -- 172.21.15.86:0/1706771243 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0b7ea0 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efedc0998b0 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7efed5600700 1 -- 172.21.15.86:0/1706771243 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7efedc0a1d30 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7efed5600700 1 -- 172.21.15.86:0/1706771243 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efedc004590 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 10 client.25442 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565fbf4fea0 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7efed5600700 1 -- 172.21.15.86:0/1706771243 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efedc0a5070 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7efed5600700 1 -- 172.21.15.86:0/1706771243 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efedc065340 con 0x5565fc037c20 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 10 client.25442 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 10 client.25442 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:11:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 20 client.25442 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc09f5b0 0x5565fc0a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 4 client.25442 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.211+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5565fbdd8cf0 con 0x5565fc09f5b0 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7efeeaa00700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc09f5b0 0x5565fc0a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7efed5600700 10 client.25442 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7efeeaa00700 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc09f5b0 0x5565fc0a19a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=400fe4d026afdc6f server_cookie=cd603a241b949d8f in_seq=0 out_seq=0 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7efed5600700 1 -- 172.21.15.86:0/1706771243 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5565fbdd8cf0 con 0x5565fc09f5b0 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7efed5600700 10 client.25442 handle_command_reply: tid=0 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7eff05e89b80 1 client.25442 shutdown 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 msgr2=0x5565fc044cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.212+0000 7eff05e89b80 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc044cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efedc000c00 tx=0x7efedc004750 comp rx=0 tx=0).stop 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 shutdown_connections 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5565fc09f5b0 0x5565fc0a19a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565fc039420 0x5565fc048190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5565fc037c20 0x5565fc044cd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efed5600700 0 client.25442 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 --2- 172.21.15.86:0/1706771243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565fc038580 0x5565fc045210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 >> 172.21.15.86:0/1706771243 conn(0x5565fbfad220 msgr2=0x5565fc0418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 shutdown_connections 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 1 -- 172.21.15.86:0/1706771243 wait complete. 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7eff05e89b80 20 client.25442 trim_cache size 0 max 16384 2024-03-22T19:11:24.220 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efed00a0a90 msgr2=0x7efed00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efed00a0a90 0x7efed00a2f50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efee400b0d0 tx=0x7efee400b040 comp rx=0 tx=0).stop 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 msgr2=0x7efeec198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 0x7efeec198ff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e970 tx=0x7efedc00ec80 comp rx=0 tx=0).stop 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 shutdown_connections 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efed00a0a90 0x7efed00a2f50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec1028e0 0x7efeec1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeec100d50 0x7efeec198ff0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 --2- 172.21.15.86:0/4070199293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efeec101f40 0x7efeec199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.213+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 >> 172.21.15.86:0/4070199293 conn(0x7efeec0fc2c0 msgr2=0x7efeec072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.214+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 shutdown_connections 2024-03-22T19:11:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.214+0000 7efef2400700 1 -- 172.21.15.86:0/4070199293 wait complete. 2024-03-22T19:11:24.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:11:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 -- 172.21.15.86:0/3298901147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 msgr2=0x7fc370073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 --2- 172.21.15.86:0/3298901147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc370073240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc364009990 tx=0x7fc364009ca0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 -- 172.21.15.86:0/3298901147 shutdown_connections 2024-03-22T19:11:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 --2- 172.21.15.86:0/3298901147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370108c10 0x7fc37010b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 --2- 172.21.15.86:0/3298901147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc370073240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 --2- 172.21.15.86:0/3298901147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 -- 172.21.15.86:0/3298901147 >> 172.21.15.86:0/3298901147 conn(0x7fc3700fc2c0 msgr2=0x7fc3700fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 -- 172.21.15.86:0/3298901147 shutdown_connections 2024-03-22T19:11:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.427+0000 7fc377c00700 1 -- 172.21.15.86:0/3298901147 wait complete. 2024-03-22T19:11:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.428+0000 7fc377c00700 1 Processor -- start 2024-03-22T19:11:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 -- start start 2024-03-22T19:11:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc3701993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370108c10 0x7fc3701a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3701a09b0 con 0x7fc370074df0 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3701a0af0 con 0x7fc370073810 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc377c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3701a0c80 con 0x7fc370108c10 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc3701993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc375200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc375200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39992/0 (socket says 172.21.15.86:39992) 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc375200700 1 -- 172.21.15.86:0/970571136 learned_addr learned my addr 172.21.15.86:0/970571136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc3701993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38898/0 (socket says 172.21.15.86:38898) 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc375c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370108c10 0x7fc3701a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.429+0000 7fc36fe00700 1 -- 172.21.15.86:0/970571136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370108c10 msgr2=0x7fc3701a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36fe00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370108c10 0x7fc3701a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36fe00700 1 -- 172.21.15.86:0/970571136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 msgr2=0x7fc370198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36fe00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36fe00700 1 -- 172.21.15.86:0/970571136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640095f0 con 0x7fc370074df0 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc375200700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36fe00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc3701993c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00e9c0 tx=0x7fc35c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36d600700 1 -- 172.21.15.86:0/970571136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c004220 con 0x7fc370074df0 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36d600700 1 -- 172.21.15.86:0/970571136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c004380 con 0x7fc370074df0 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3701a0f60 con 0x7fc370074df0 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc36d600700 1 -- 172.21.15.86:0/970571136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c003be0 con 0x7fc370074df0 2024-03-22T19:11:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.430+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3701a1430 con 0x7fc370074df0 2024-03-22T19:11:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.433+0000 7fc36d600700 1 -- 172.21.15.86:0/970571136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fc35c003d40 con 0x7fc370074df0 2024-03-22T19:11:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 Processor -- start 2024-03-22T19:11:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 -- start start 2024-03-22T19:11:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc36d600700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc3580a0b10 0x7fc3580a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcbb420 0x56028dcbb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dc12990 con 0x56028dcbb420 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.435+0000 7fc38b762b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dc10350 con 0x56028dcba580 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc38b762b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dc0bc40 con 0x56028dcb9c20 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36d600700 1 -- 172.21.15.86:0/970571136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc35c014070 con 0x7fc370074df0 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc375c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc3580a0b10 0x7fc3580a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40002/0 (socket says 172.21.15.86:40002) 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 -- 172.21.15.86:0/1364175245 learned_addr learned my addr 172.21.15.86:0/1364175245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc375200700 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc375c00700 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcbb420 0x56028dcbb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 -- 172.21.15.86:0/1364175245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 msgr2=0x56028dcba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 -- 172.21.15.86:0/1364175245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcbb420 msgr2=0x56028dcbb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcbb420 0x56028dcbb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 -- 172.21.15.86:0/1364175245 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56028da8d3b0 con 0x56028dcba580 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.436+0000 7fc36fe00700 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbae40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc35c096330 tx=0x7fc35c0997c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72bfde0bda18116b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc375c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc3580a0b10 0x7fc3580a2fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x56028dcbac00 tx=0x7fc368009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc362000700 1 -- 172.21.15.86:0/1364175245 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0b2470 con 0x56028dcba580 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc362000700 1 -- 172.21.15.86:0/1364175245 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c0044f0 con 0x56028dcba580 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc362000700 1 -- 172.21.15.86:0/1364175245 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0a1d40 con 0x56028dcba580 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 -- 172.21.15.86:0/1364175245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 msgr2=0x56028dcbae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbae40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc35c096330 tx=0x7fc35c0997c0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 -- 172.21.15.86:0/1364175245 shutdown_connections 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcbb420 0x56028dcbb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 --2- 172.21.15.86:0/1364175245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbae40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.437+0000 7fc38b762b80 1 -- 172.21.15.86:0/1364175245 >> 172.21.15.86:0/1364175245 conn(0x56028dc2f220 msgr2=0x56028dcb4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.438+0000 7fc38b762b80 1 -- 172.21.15.86:0/1364175245 shutdown_connections 2024-03-22T19:11:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.438+0000 7fc38b762b80 1 -- 172.21.15.86:0/1364175245 wait complete. 2024-03-22T19:11:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.438+0000 7fc38b762b80 1 -- start start 2024-03-22T19:11:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.438+0000 7fc38b762b80 1 Processor -- start 2024-03-22T19:11:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcb9c20 0x56028dcc3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dccafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dba7d80 con 0x56028dcba580 2024-03-22T19:11:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dba7a20 con 0x56028dcb9c20 2024-03-22T19:11:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028da8f6a0 con 0x56028dcbb420 2024-03-22T19:11:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc38b762b80 10 client.0 _ll_register_callbacks cb 0x56028da133a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38912/0 (socket says 172.21.15.86:38912) 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.439+0000 7fc36fe00700 1 -- 172.21.15.86:0/2040622465 learned_addr learned my addr 172.21.15.86:0/2040622465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc375200700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcb9c20 0x56028dcc3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc375c00700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dccafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc36fe00700 1 -- 172.21.15.86:0/2040622465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 msgr2=0x56028dccafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc36fe00700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dccafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc36fe00700 1 -- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcb9c20 msgr2=0x56028dcc3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc36fe00700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcb9c20 0x56028dcc3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc36fe00700 1 -- 172.21.15.86:0/2040622465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3701a1430 con 0x56028dcba580 2024-03-22T19:11:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc36fe00700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc3f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc35c095dc0 tx=0x7fc35c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c74f206b1a7fa47 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc356a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc356a00700 1 -- 172.21.15.86:0/2040622465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0b2470 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc356a00700 1 -- 172.21.15.86:0/2040622465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc35c099fc0 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56028dcacad0 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc356a00700 1 -- 172.21.15.86:0/2040622465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c0a1d40 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.440+0000 7fc38b762b80 10 client.15429 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56028da39b60 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc356a00700 1 -- 172.21.15.86:0/2040622465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56028dcacad0 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc356a00700 1 -- 172.21.15.86:0/2040622465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc35c098ed0 con 0x56028dcba580 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 10 client.15429 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 20 client.15429 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dccf330 0x56028dcd1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 4 client.15429 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56028dc04700 con 0x56028dccf330 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.441+0000 7fc375200700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dccf330 0x56028dcd1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.442+0000 7fc356a00700 10 client.15429 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.442+0000 7fc375200700 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dccf330 0x56028dcd1750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f68950d3881166a8 server_cookie=1854a4bc58c46e99 in_seq=0 out_seq=0 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc356a00700 1 -- 172.21.15.86:0/2040622465 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56028dc04700 con 0x56028dccf330 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc356a00700 10 client.15429 handle_command_reply: tid=0 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 client.15429 shutdown 2024-03-22T19:11:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 msgr2=0x56028dcc3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc3f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc35c095dc0 tx=0x7fc35c0047c0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 shutdown_connections 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dccf330 0x56028dcd1750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dccafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcb9c20 0x56028dcc3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc356a00700 0 client.15429 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2040622465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc3f70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 >> 172.21.15.86:0/2040622465 conn(0x56028dc2f220 msgr2=0x56028dcb41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 shutdown_connections 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 1 -- 172.21.15.86:0/2040622465 wait complete. 2024-03-22T19:11:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.443+0000 7fc38b762b80 20 client.15429 trim_cache size 0 max 16384 2024-03-22T19:11:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.456+0000 7fc38b762b80 1 Processor -- start 2024-03-22T19:11:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.456+0000 7fc38b762b80 1 -- start start 2024-03-22T19:11:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 0x56028dcc8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dcc1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc38b762b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dcaf210 con 0x56028dcb9c20 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc38b762b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dccc130 con 0x56028dcba580 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc38b762b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dcc21b0 con 0x56028dcbb420 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 0x56028dcc8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 0x56028dcc8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38928/0 (socket says 172.21.15.86:38928) 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 -- 172.21.15.86:0/3411695880 learned_addr learned my addr 172.21.15.86:0/3411695880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc36fe00700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 -- 172.21.15.86:0/3411695880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 msgr2=0x56028dcc1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375c00700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dcc1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dcc1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 -- 172.21.15.86:0/3411695880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 msgr2=0x56028dcbb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 -- 172.21.15.86:0/3411695880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56028dbd1ea0 con 0x56028dcb9c20 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc36fe00700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375c00700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dcc1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.457+0000 7fc375200700 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 0x56028dcc8f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc364000c00 tx=0x7fc36400b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f903f85d0364d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc362a00700 1 -- 172.21.15.86:0/3411695880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36401d040 con 0x56028dcb9c20 2024-03-22T19:11:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc362a00700 1 -- 172.21.15.86:0/3411695880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36401e040 con 0x56028dcb9c20 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc362a00700 1 -- 172.21.15.86:0/3411695880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640056d0 con 0x56028dcb9c20 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 -- 172.21.15.86:0/3411695880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 msgr2=0x56028dcc8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 0x56028dcc8f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc364000c00 tx=0x7fc36400b750 comp rx=0 tx=0).stop 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 -- 172.21.15.86:0/3411695880 shutdown_connections 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcbb420 0x56028dcc1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcb9c20 0x56028dcc8f60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 --2- 172.21.15.86:0/3411695880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcba580 0x56028dcbb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 -- 172.21.15.86:0/3411695880 >> 172.21.15.86:0/3411695880 conn(0x56028dc2f220 msgr2=0x56028dcc2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 -- 172.21.15.86:0/3411695880 shutdown_connections 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.458+0000 7fc38b762b80 1 -- 172.21.15.86:0/3411695880 wait complete. 2024-03-22T19:11:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.459+0000 7fc38b762b80 1 -- start start 2024-03-22T19:11:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.459+0000 7fc38b762b80 1 Processor -- start 2024-03-22T19:11:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcc8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcbb420 0x56028dcc7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dcc7870 con 0x56028dcba580 2024-03-22T19:11:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dcc79e0 con 0x56028dcbb420 2024-03-22T19:11:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028dcc7ba0 con 0x56028dcb9c20 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc38b762b80 10 client.0 _ll_register_callbacks cb 0x7fc35c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38938/0 (socket says 172.21.15.86:38938) 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 -- 172.21.15.86:0/2117056389 learned_addr learned my addr 172.21.15.86:0/2117056389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc375200700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcc8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc375c00700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcbb420 0x56028dcc7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 -- 172.21.15.86:0/2117056389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 msgr2=0x56028dcc8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcc8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 -- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcbb420 msgr2=0x56028dcc7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcbb420 0x56028dcc7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.460+0000 7fc36fe00700 1 -- 172.21.15.86:0/2117056389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37010bcc0 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc36fe00700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc8f50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00cf50 tx=0x7fc35c095d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59606f42aed0d4a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc356a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc356a00700 1 -- 172.21.15.86:0/2117056389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0a1950 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc356a00700 1 -- 172.21.15.86:0/2117056389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc35c099fc0 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc35c0044f0 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc356a00700 1 -- 172.21.15.86:0/2117056389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c0b2470 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc38b762b80 10 client.15435 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56028da91cd0 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc356a00700 1 -- 172.21.15.86:0/2117056389 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c0044f0 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc356a00700 1 -- 172.21.15.86:0/2117056389 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc35c098db0 con 0x56028dcba580 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc38b762b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc38b762b80 10 client.15435 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:11:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.461+0000 7fc38b762b80 20 client.15435 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.462+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dd21ca0 0x56028dd240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.462+0000 7fc38b762b80 4 client.15435 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.462+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56028da5acf0 con 0x56028dd21ca0 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.462+0000 7fc375200700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dd21ca0 0x56028dd240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc356a00700 10 client.15435 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc375200700 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dd21ca0 0x56028dd240c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2574f8762ef4ddd4 server_cookie=b3ec059bc91b38d7 in_seq=0 out_seq=0 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc356a00700 1 -- 172.21.15.86:0/2117056389 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56028da5acf0 con 0x56028dd21ca0 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc356a00700 10 client.15435 handle_command_reply: tid=0 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 client.15435 shutdown 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 msgr2=0x56028dcc8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc8f50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00cf50 tx=0x7fc35c095d30 comp rx=0 tx=0).stop 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 shutdown_connections 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56028dcbb420 0x56028dcc7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x56028dd21ca0 0x56028dd240c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56028dcb9c20 0x56028dcc8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 --2- 172.21.15.86:0/2117056389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56028dcba580 0x56028dcc8f50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 >> 172.21.15.86:0/2117056389 conn(0x56028dc2f220 msgr2=0x56028dcbe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc356a00700 0 client.15435 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 shutdown_connections 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.463+0000 7fc38b762b80 1 -- 172.21.15.86:0/2117056389 wait complete. 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc38b762b80 20 client.15435 trim_cache size 0 max 16384 2024-03-22T19:11:24.472 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc3580a0b10 msgr2=0x7fc3580a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc3580a0b10 0x7fc3580a2fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x56028dcbac00 tx=0x7fc368009920 comp rx=0 tx=0).stop 2024-03-22T19:11:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 msgr2=0x7fc3701993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc3701993c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00e9c0 tx=0x7fc35c00ecd0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 shutdown_connections 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc3580a0b10 0x7fc3580a2fd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370108c10 0x7fc3701a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc370073810 0x7fc370198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 --2- 172.21.15.86:0/970571136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc370074df0 0x7fc3701993c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 >> 172.21.15.86:0/970571136 conn(0x7fc3700fc2c0 msgr2=0x7fc370104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 shutdown_connections 2024-03-22T19:11:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.464+0000 7fc377c00700 1 -- 172.21.15.86:0/970571136 wait complete. 2024-03-22T19:11:24.474 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:24 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/476563881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:11:24.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:24 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/476563881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:11:24.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:11:24.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:24 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/476563881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:11:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.678+0000 7fdd64e00700 1 -- 172.21.15.86:0/494968570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60101f40 msgr2=0x7fdd601023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.678+0000 7fdd64e00700 1 --2- 172.21.15.86:0/494968570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60101f40 0x7fdd601023a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0099c0 tx=0x7fdd4c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 -- 172.21.15.86:0/494968570 shutdown_connections 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 --2- 172.21.15.86:0/494968570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd601028e0 0x7fdd60109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 --2- 172.21.15.86:0/494968570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd60100d50 0x7fdd60101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 --2- 172.21.15.86:0/494968570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60101f40 0x7fdd601023a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 -- 172.21.15.86:0/494968570 >> 172.21.15.86:0/494968570 conn(0x7fdd600fc2c0 msgr2=0x7fdd600fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 -- 172.21.15.86:0/494968570 shutdown_connections 2024-03-22T19:11:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.679+0000 7fdd64e00700 1 -- 172.21.15.86:0/494968570 wait complete. 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 Processor -- start 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 -- start start 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60100d50 0x7fdd60074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd60101f40 0x7fdd600731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 0x7fdd600736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60073e80 con 0x7fdd601028e0 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60073ff0 con 0x7fdd60100d50 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.680+0000 7fdd64e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601a0d50 con 0x7fdd60101f40 2024-03-22T19:11:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 0x7fdd600736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 0x7fdd600736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38960/0 (socket says 172.21.15.86:38960) 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 -- 172.21.15.86:0/499913464 learned_addr learned my addr 172.21.15.86:0/499913464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5de00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60100d50 0x7fdd60074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5d400700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd60101f40 0x7fdd600731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 -- 172.21.15.86:0/499913464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd60101f40 msgr2=0x7fdd600731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd60101f40 0x7fdd600731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 -- 172.21.15.86:0/499913464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60100d50 msgr2=0x7fdd60074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60100d50 0x7fdd60074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 -- 172.21.15.86:0/499913464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50009520 con 0x7fdd601028e0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd5e800700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 0x7fdd600736e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800ef30 tx=0x7fdd5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.681+0000 7fdd56a00700 1 -- 172.21.15.86:0/499913464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58018070 con 0x7fdd601028e0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.682+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c0095f0 con 0x7fdd601028e0 2024-03-22T19:11:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.682+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd601a1280 con 0x7fdd601028e0 2024-03-22T19:11:24.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.682+0000 7fdd56a00700 1 -- 172.21.15.86:0/499913464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd5800f040 con 0x7fdd601028e0 2024-03-22T19:11:24.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.682+0000 7fdd56a00700 1 -- 172.21.15.86:0/499913464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800cc10 con 0x7fdd601028e0 2024-03-22T19:11:24.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.685+0000 7fdd56a00700 1 -- 172.21.15.86:0/499913464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fdd5800cd70 con 0x7fdd601028e0 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 Processor -- start 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd56a00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd440a0b10 0x7fdd440a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 -- start start 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd56a00700 1 -- 172.21.15.86:0/499913464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fdd580cb910 con 0x7fdd601028e0 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd5d400700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd440a0b10 0x7fdd440a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb260040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb260e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb2618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb1b8990 con 0x563cbb261420 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb1b6350 con 0x563cbb260580 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.687+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb1b1c40 con 0x563cbb25fc20 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb260e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb2618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb260e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40062/0 (socket says 172.21.15.86:40062) 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5d400700 1 -- 172.21.15.86:0/1434050658 learned_addr learned my addr 172.21.15.86:0/1434050658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5de00700 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb260040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 -- 172.21.15.86:0/1434050658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 msgr2=0x563cbb260040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb260040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 -- 172.21.15.86:0/1434050658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 msgr2=0x563cbb260e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb260e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 -- 172.21.15.86:0/1434050658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbb0333b0 con 0x563cbb261420 2024-03-22T19:11:24.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5d400700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd440a0b10 0x7fdd440a2fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c00bb40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd5e800700 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb2618a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000f80 tx=0x7fdd58098a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc603986f17878e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.688+0000 7fdd4b400700 1 -- 172.21.15.86:0/1434050658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58099020 con 0x563cbb261420 2024-03-22T19:11:24.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd4b400700 1 -- 172.21.15.86:0/1434050658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd5809ce50 con 0x563cbb261420 2024-03-22T19:11:24.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd4b400700 1 -- 172.21.15.86:0/1434050658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58099480 con 0x563cbb261420 2024-03-22T19:11:24.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 -- 172.21.15.86:0/1434050658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 msgr2=0x563cbb2618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb2618a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000f80 tx=0x7fdd58098a80 comp rx=0 tx=0).stop 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 -- 172.21.15.86:0/1434050658 shutdown_connections 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb2618a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb260040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/1434050658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb260e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 -- 172.21.15.86:0/1434050658 >> 172.21.15.86:0/1434050658 conn(0x563cbb1d5220 msgr2=0x563cbb25abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 -- 172.21.15.86:0/1434050658 shutdown_connections 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.689+0000 7fdd788c3b80 1 -- 172.21.15.86:0/1434050658 wait complete. 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.690+0000 7fdd788c3b80 1 -- start start 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.690+0000 7fdd788c3b80 1 Processor -- start 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.690+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb269b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.690+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb26a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb271070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb14dd80 con 0x563cbb261420 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb14da20 con 0x563cbb260580 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb0356a0 con 0x563cbb25fc20 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd788c3b80 10 client.0 _ll_register_callbacks cb 0x563cbafb93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb271070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb271070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38988/0 (socket says 172.21.15.86:38988) 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb269b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 -- 172.21.15.86:0/4232429922 learned_addr learned my addr 172.21.15.86:0/4232429922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5d400700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb26a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 -- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 msgr2=0x563cbb269b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb269b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 -- 172.21.15.86:0/4232429922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 msgr2=0x563cbb26a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb26a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 -- 172.21.15.86:0/4232429922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd601a1280 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5d400700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb26a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd5e800700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb271070 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdd58095520 tx=0x7fdd580997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20723277d1119e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd3fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.691+0000 7fdd3fe00700 1 -- 172.21.15.86:0/4232429922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58099480 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd3fe00700 1 -- 172.21.15.86:0/4232429922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fdd580a1410 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cbb252ad0 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd3fe00700 1 -- 172.21.15.86:0/4232429922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd580a0f80 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 10 client.15447 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563cbafdfb60 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd3fe00700 1 -- 172.21.15.86:0/4232429922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd580b7020 con 0x563cbb261420 2024-03-22T19:11:24.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd3fe00700 1 -- 172.21.15.86:0/4232429922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdd58082c40 con 0x563cbb261420 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 10 client.15447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 20 client.15447 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb275360 0x563cbb277780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 4 client.15447 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.692+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cbb1aa700 con 0x563cbb275360 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.693+0000 7fdd5de00700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb275360 0x563cbb277780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.693+0000 7fdd3fe00700 10 client.15447 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.693+0000 7fdd5de00700 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb275360 0x563cbb277780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67d0ec53eb54ff04 server_cookie=ef0ad9efa6addf0 in_seq=0 out_seq=0 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd3fe00700 1 -- 172.21.15.86:0/4232429922 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563cbb1aa700 con 0x563cbb275360 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd3fe00700 10 client.15447 handle_command_reply: tid=0 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 client.15447 shutdown 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 msgr2=0x563cbb271070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb271070 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdd58095520 tx=0x7fdd580997e0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 shutdown_connections 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb275360 0x563cbb277780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb261420 0x563cbb271070 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb269b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd3fe00700 0 client.15447 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/4232429922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb260580 0x563cbb26a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 >> 172.21.15.86:0/4232429922 conn(0x563cbb1d5220 msgr2=0x563cbb25a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 shutdown_connections 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 1 -- 172.21.15.86:0/4232429922 wait complete. 2024-03-22T19:11:24.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.694+0000 7fdd788c3b80 20 client.15447 trim_cache size 0 max 16384 2024-03-22T19:11:24.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.707+0000 7fdd788c3b80 1 Processor -- start 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.707+0000 7fdd788c3b80 1 -- start start 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb271720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb25ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb255210 con 0x563cbb260580 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb2696b0 con 0x563cbb261420 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb2575e0 con 0x563cbb25fc20 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb271720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb271720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38994/0 (socket says 172.21.15.86:38994) 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5d400700 1 -- 172.21.15.86:0/3846195182 learned_addr learned my addr 172.21.15.86:0/3846195182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5e800700 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb25ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 -- 172.21.15.86:0/3846195182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 msgr2=0x563cbb25ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb25ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 -- 172.21.15.86:0/3846195182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 msgr2=0x563cbb271720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb271720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 -- 172.21.15.86:0/3846195182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbb177ea0 con 0x563cbb25fc20 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5d400700 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb271720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.708+0000 7fdd5de00700 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26f020 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x563cbb275740 tx=0x7fdd50009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98742afa839503d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd4be00700 1 -- 172.21.15.86:0/3846195182 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50017040 con 0x563cbb25fc20 2024-03-22T19:11:24.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd4be00700 1 -- 172.21.15.86:0/3846195182 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd50018040 con 0x563cbb25fc20 2024-03-22T19:11:24.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd4be00700 1 -- 172.21.15.86:0/3846195182 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50005280 con 0x563cbb25fc20 2024-03-22T19:11:24.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 -- 172.21.15.86:0/3846195182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 msgr2=0x563cbb26f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26f020 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x563cbb275740 tx=0x7fdd50009260 comp rx=0 tx=0).stop 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 -- 172.21.15.86:0/3846195182 shutdown_connections 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb25ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26f020 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/3846195182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb271720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 -- 172.21.15.86:0/3846195182 >> 172.21.15.86:0/3846195182 conn(0x563cbb1d5220 msgr2=0x563cbb256a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 -- 172.21.15.86:0/3846195182 shutdown_connections 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.709+0000 7fdd788c3b80 1 -- 172.21.15.86:0/3846195182 wait complete. 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.710+0000 7fdd788c3b80 1 -- start start 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.710+0000 7fdd788c3b80 1 Processor -- start 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb26d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb26e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb26ea70 con 0x563cbb260580 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb26ebe0 con 0x563cbb261420 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb26ed50 con 0x563cbb25fc20 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd788c3b80 10 client.0 _ll_register_callbacks cb 0x7fdd4c005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb26d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb26d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38996/0 (socket says 172.21.15.86:38996) 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 -- 172.21.15.86:0/877109471 learned_addr learned my addr 172.21.15.86:0/877109471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5e800700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb26e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5de00700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 -- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 msgr2=0x563cbb26cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 -- 172.21.15.86:0/877109471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 msgr2=0x563cbb26e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb26e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.711+0000 7fdd5d400700 1 -- 172.21.15.86:0/877109471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60109ae0 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd5d400700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb26d520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009990 tx=0x7fdd4c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b398e4cf28bca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd3fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd3fe00700 1 -- 172.21.15.86:0/877109471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c005240 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd3fe00700 1 -- 172.21.15.86:0/877109471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fdd4c0053a0 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd5809ce50 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd3fe00700 1 -- 172.21.15.86:0/877109471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4c005560 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd788c3b80 10 client.15453 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fdd50009e60 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd3fe00700 1 -- 172.21.15.86:0/877109471 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c043020 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd3fe00700 1 -- 172.21.15.86:0/877109471 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdd4c034430 con 0x563cbb260580 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd788c3b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.712+0000 7fdd788c3b80 10 client.15453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd788c3b80 20 client.15453 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb2c8130 0x563cbb2ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd788c3b80 4 client.15453 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563cbb000cf0 con 0x563cbb2c8130 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd5de00700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb2c8130 0x563cbb2ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd3fe00700 10 client.15453 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.713+0000 7fdd5de00700 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb2c8130 0x563cbb2ca550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=897a6e9e64016de1 server_cookie=c3d259433e7e6fb8 in_seq=0 out_seq=0 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd3fe00700 1 -- 172.21.15.86:0/877109471 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cbb000cf0 con 0x563cbb2c8130 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd3fe00700 10 client.15453 handle_command_reply: tid=0 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 client.15453 shutdown 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 msgr2=0x563cbb26d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb26d520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009990 tx=0x7fdd4c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 shutdown_connections 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x563cbb2c8130 0x563cbb2ca550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563cbb261420 0x563cbb26e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd3fe00700 0 client.15453 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cbb25fc20 0x563cbb26cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 --2- 172.21.15.86:0/877109471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563cbb260580 0x563cbb26d520 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 >> 172.21.15.86:0/877109471 conn(0x563cbb1d5220 msgr2=0x563cbb25aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 shutdown_connections 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 1 -- 172.21.15.86:0/877109471 wait complete. 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.714+0000 7fdd788c3b80 20 client.15453 trim_cache size 0 max 16384 2024-03-22T19:11:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd440a0b10 msgr2=0x7fdd440a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd440a0b10 0x7fdd440a2fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c00bb40 comp rx=0 tx=0).stop 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 msgr2=0x7fdd600736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 0x7fdd600736e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800ef30 tx=0x7fdd5800c3a0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 shutdown_connections 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fdd440a0b10 0x7fdd440a2fd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601028e0 0x7fdd600736e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60100d50 0x7fdd60074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 --2- 172.21.15.86:0/499913464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd60101f40 0x7fdd600731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 >> 172.21.15.86:0/499913464 conn(0x7fdd600fc2c0 msgr2=0x7fdd60105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 shutdown_connections 2024-03-22T19:11:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.715+0000 7fdd64e00700 1 -- 172.21.15.86:0/499913464 wait complete. 2024-03-22T19:11:24.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:11:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 -- 172.21.15.86:0/2435358138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 msgr2=0x7fa3c01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 --2- 172.21.15.86:0/2435358138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c01023a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0099c0 tx=0x7fa3ac009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 -- 172.21.15.86:0/2435358138 shutdown_connections 2024-03-22T19:11:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 --2- 172.21.15.86:0/2435358138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c01028e0 0x7fa3c0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 --2- 172.21.15.86:0/2435358138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c0100d50 0x7fa3c0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 --2- 172.21.15.86:0/2435358138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c01023a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 -- 172.21.15.86:0/2435358138 >> 172.21.15.86:0/2435358138 conn(0x7fa3c00fc2c0 msgr2=0x7fa3c00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.926+0000 7fa3c7200700 1 -- 172.21.15.86:0/2435358138 shutdown_connections 2024-03-22T19:11:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.927+0000 7fa3c7200700 1 -- 172.21.15.86:0/2435358138 wait complete. 2024-03-22T19:11:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.927+0000 7fa3c7200700 1 Processor -- start 2024-03-22T19:11:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 -- start start 2024-03-22T19:11:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c0100d50 0x7fa3c0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c01028e0 0x7fa3c01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01a0a30 con 0x7fa3c0101f40 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01a0b70 con 0x7fa3c0100d50 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c7200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01a0d00 con 0x7fa3c01028e0 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c0199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39020/0 (socket says 172.21.15.86:39020) 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3bf400700 1 -- 172.21.15.86:0/455540659 learned_addr learned my addr 172.21.15.86:0/455540659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3c5200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c01028e0 0x7fa3c01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.928+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c0100d50 0x7fa3c0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bf400700 1 -- 172.21.15.86:0/455540659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c01028e0 msgr2=0x7fa3c01a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bf400700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c01028e0 0x7fa3c01a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bf400700 1 -- 172.21.15.86:0/455540659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c0100d50 msgr2=0x7fa3c0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bf400700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c0100d50 0x7fa3c0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bf400700 1 -- 172.21.15.86:0/455540659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0095f0 con 0x7fa3c0101f40 2024-03-22T19:11:24.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bf400700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c0199440 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c010 tx=0x7fa3ac005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bcc00700 1 -- 172.21.15.86:0/455540659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac01d070 con 0x7fa3c0101f40 2024-03-22T19:11:24.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bcc00700 1 -- 172.21.15.86:0/455540659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3ac005050 con 0x7fa3c0101f40 2024-03-22T19:11:24.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c01a0f80 con 0x7fa3c0101f40 2024-03-22T19:11:24.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3bcc00700 1 -- 172.21.15.86:0/455540659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac004b30 con 0x7fa3c0101f40 2024-03-22T19:11:24.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.929+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c01a13a0 con 0x7fa3c0101f40 2024-03-22T19:11:24.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.932+0000 7fa3bcc00700 1 -- 172.21.15.86:0/455540659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa3ac004c90 con 0x7fa3c0101f40 2024-03-22T19:11:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.934+0000 7fa3dada3b80 1 Processor -- start 2024-03-22T19:11:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.934+0000 7fa3bcc00700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3a40a0b10 0x7fa3a40a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 -- start start 2024-03-22T19:11:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3bcc00700 1 -- 172.21.15.86:0/455540659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa3ac0cce20 con 0x7fa3c0101f40 2024-03-22T19:11:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3c5200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3a40a0b10 0x7fa3a40a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a43c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a43ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a43d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a394990 con 0x56207a43c580 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a392350 con 0x56207a43d420 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a38dc40 con 0x56207a43bc20 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a43c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a43c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51300/0 (socket says 172.21.15.86:51300) 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3bfe00700 1 -- 172.21.15.86:0/2684526043 learned_addr learned my addr 172.21.15.86:0/2684526043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3bf400700 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a43ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.935+0000 7fa3c5200700 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a43d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3bfe00700 1 -- 172.21.15.86:0/2684526043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 msgr2=0x56207a43d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a43d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3bfe00700 1 -- 172.21.15.86:0/2684526043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 msgr2=0x56207a43ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a43ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3bfe00700 1 -- 172.21.15.86:0/2684526043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56207a20f3b0 con 0x56207a43bc20 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3c5200700 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a43d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3c5200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3a40a0b10 0x7fa3a40a2fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8003ee0 tx=0x7fa3b8009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a43c040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009d30 tx=0x7fa3b0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feef265f91c80f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3b5600700 1 -- 172.21.15.86:0/2684526043 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0004220 con 0x56207a43bc20 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3b5600700 1 -- 172.21.15.86:0/2684526043 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3b0004380 con 0x56207a43bc20 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.936+0000 7fa3b5600700 1 -- 172.21.15.86:0/2684526043 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00198d0 con 0x56207a43bc20 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 -- 172.21.15.86:0/2684526043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 msgr2=0x56207a43c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a43c040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009d30 tx=0x7fa3b0011720 comp rx=0 tx=0).stop 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 -- 172.21.15.86:0/2684526043 shutdown_connections 2024-03-22T19:11:24.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a43d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a43c040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/2684526043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a43ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 -- 172.21.15.86:0/2684526043 >> 172.21.15.86:0/2684526043 conn(0x56207a3b1220 msgr2=0x56207a436bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 -- 172.21.15.86:0/2684526043 shutdown_connections 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.937+0000 7fa3dada3b80 1 -- 172.21.15.86:0/2684526043 wait complete. 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 -- start start 2024-03-22T19:11:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 Processor -- start 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a445b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a446060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a44d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a329d80 con 0x56207a43c580 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a329a20 con 0x56207a43d420 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a2116a0 con 0x56207a43bc20 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3dada3b80 10 client.0 _ll_register_callbacks cb 0x56207a1953a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a446060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.938+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a445b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a446060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39042/0 (socket says 172.21.15.86:39042) 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a445b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51316/0 (socket says 172.21.15.86:51316) 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 -- 172.21.15.86:0/1500282849 learned_addr learned my addr 172.21.15.86:0/1500282849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3c5200700 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a44d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 -- 172.21.15.86:0/1500282849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 msgr2=0x56207a445b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a445b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 -- 172.21.15.86:0/1500282849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 msgr2=0x56207a44d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a44d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 -- 172.21.15.86:0/1500282849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b0009960 con 0x56207a43c580 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3bf400700 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a446060 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac09a4c0 tx=0x7fa3ac0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d147285068363fed server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3aa000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3aa000700 1 -- 172.21.15.86:0/1500282849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac099c20 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3aa000700 1 -- 172.21.15.86:0/1500282849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa3ac09ac30 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56207a42ead0 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3aa000700 1 -- 172.21.15.86:0/1500282849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3ac09e120 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.939+0000 7fa3dada3b80 10 client.15465 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56207a1bbb60 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3aa000700 1 -- 172.21.15.86:0/1500282849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56207a42ead0 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3aa000700 1 -- 172.21.15.86:0/1500282849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3ac0a25b0 con 0x56207a43c580 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 10 client.15465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 20 client.15465 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a451480 0x56207a453870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 4 client.15465 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56207a386700 con 0x56207a451480 2024-03-22T19:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.940+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a451480 0x56207a453870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a451480 0x56207a453870 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bb65a5b52f5df9 server_cookie=6ac8dbe7a698bd32 in_seq=0 out_seq=0 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3aa000700 10 client.15465 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3aa000700 1 -- 172.21.15.86:0/1500282849 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56207a386700 con 0x56207a451480 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3aa000700 10 client.15465 handle_command_reply: tid=0 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3dada3b80 1 client.15465 shutdown 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 msgr2=0x56207a446060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.941+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a446060 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac09a4c0 tx=0x7fa3ac0057a0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 shutdown_connections 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a451480 0x56207a453870 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43d420 0x56207a44d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43bc20 0x56207a445b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3aa000700 0 client.15465 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1500282849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a446060 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 >> 172.21.15.86:0/1500282849 conn(0x56207a3b1220 msgr2=0x56207a4361f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 shutdown_connections 2024-03-22T19:11:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1500282849 wait complete. 2024-03-22T19:11:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.942+0000 7fa3dada3b80 20 client.15465 trim_cache size 0 max 16384 2024-03-22T19:11:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 Processor -- start 2024-03-22T19:11:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 -- start start 2024-03-22T19:11:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 0x56207a44afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43c580 0x56207a44d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a445030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a4335e0 con 0x56207a43bc20 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a437930 con 0x56207a43c580 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.955+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a4383b0 con 0x56207a43d420 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 0x56207a44afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 0x56207a44afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39054/0 (socket says 172.21.15.86:39054) 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 -- 172.21.15.86:0/4131442588 learned_addr learned my addr 172.21.15.86:0/4131442588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3c5200700 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a445030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bf400700 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43c580 0x56207a44d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 -- 172.21.15.86:0/4131442588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 msgr2=0x56207a445030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a445030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 -- 172.21.15.86:0/4131442588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43c580 msgr2=0x56207a44d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43c580 0x56207a44d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 -- 172.21.15.86:0/4131442588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56207a353ea0 con 0x56207a43bc20 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.956+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 0x56207a44afb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009d30 tx=0x7fa3b0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369b03049b5ae81c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3b6000700 1 -- 172.21.15.86:0/4131442588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0029040 con 0x56207a43bc20 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3b6000700 1 -- 172.21.15.86:0/4131442588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3b0003b40 con 0x56207a43bc20 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3b6000700 1 -- 172.21.15.86:0/4131442588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00052a0 con 0x56207a43bc20 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 -- 172.21.15.86:0/4131442588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 msgr2=0x56207a44afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 0x56207a44afb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009d30 tx=0x7fa3b0011720 comp rx=0 tx=0).stop 2024-03-22T19:11:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 -- 172.21.15.86:0/4131442588 shutdown_connections 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a445030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43bc20 0x56207a44afb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/4131442588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43c580 0x56207a44d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 -- 172.21.15.86:0/4131442588 >> 172.21.15.86:0/4131442588 conn(0x56207a3b1220 msgr2=0x56207a443940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 -- 172.21.15.86:0/4131442588 shutdown_connections 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.957+0000 7fa3dada3b80 1 -- 172.21.15.86:0/4131442588 wait complete. 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 -- start start 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 Processor -- start 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43bc20 0x56207a44aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a44af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a4491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a449920 con 0x56207a43c580 2024-03-22T19:11:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a449a90 con 0x56207a43bc20 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207a449c50 con 0x56207a43d420 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.958+0000 7fa3dada3b80 10 client.0 _ll_register_callbacks cb 0x56207a1953a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a44af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43bc20 0x56207a44aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a44af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39060/0 (socket says 172.21.15.86:39060) 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bf400700 1 -- 172.21.15.86:0/1747194047 learned_addr learned my addr 172.21.15.86:0/1747194047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3c5200700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a4491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 -- 172.21.15.86:0/1747194047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 msgr2=0x56207a4491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a4491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 -- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 msgr2=0x56207a44af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a44af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 -- 172.21.15.86:0/1747194047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56207a20f3b0 con 0x56207a43bc20 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bf400700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a44af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3bfe00700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43bc20 0x56207a44aa40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000c00 tx=0x7fa3b0011a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c2e0877b3df761 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.959+0000 7fa3aa000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3aa000700 1 -- 172.21.15.86:0/1747194047 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0029070 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3b0004380 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3aa000700 1 -- 172.21.15.86:0/1747194047 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa3b0003df0 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3aa000700 1 -- 172.21.15.86:0/1747194047 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3b0003880 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3dada3b80 10 client.25493 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56207a213cd0 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3aa000700 1 -- 172.21.15.86:0/1747194047 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3b0004380 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.960+0000 7fa3aa000700 1 -- 172.21.15.86:0/1747194047 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3b0030020 con 0x56207a43bc20 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 10 client.25493 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 10 client.25493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 20 client.25493 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a4a4070 0x56207a4a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 4 client.25493 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:24.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56207a1dccf0 con 0x56207a4a4070 2024-03-22T19:11:24.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3bf400700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a4a4070 0x56207a4a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3aa000700 10 client.25493 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3bf400700 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a4a4070 0x56207a4a6460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26592a563670e97e server_cookie=3cf981a5cd782dd4 in_seq=0 out_seq=0 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3aa000700 1 -- 172.21.15.86:0/1747194047 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56207a1dccf0 con 0x56207a4a4070 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3aa000700 10 client.25493 handle_command_reply: tid=0 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.961+0000 7fa3dada3b80 1 client.25493 shutdown 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43bc20 msgr2=0x56207a44aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43bc20 0x56207a44aa40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000c00 tx=0x7fa3b0011a70 comp rx=0 tx=0).stop 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 shutdown_connections 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56207a4a4070 0x56207a4a6460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56207a43d420 0x56207a4491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56207a43bc20 0x56207a44aa40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 --2- 172.21.15.86:0/1747194047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207a43c580 0x56207a44af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3aa000700 0 client.25493 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 >> 172.21.15.86:0/1747194047 conn(0x56207a3b1220 msgr2=0x56207a4447d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 shutdown_connections 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 1 -- 172.21.15.86:0/1747194047 wait complete. 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3dada3b80 20 client.25493 trim_cache size 0 max 16384 2024-03-22T19:11:24.968 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3a40a0b10 msgr2=0x7fa3a40a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.962+0000 7fa3c7200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3a40a0b10 0x7fa3a40a2fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8003ee0 tx=0x7fa3b8009130 comp rx=0 tx=0).stop 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 msgr2=0x7fa3c0199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c0199440 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c010 tx=0x7fa3ac005f10 comp rx=0 tx=0).stop 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 shutdown_connections 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa3a40a0b10 0x7fa3a40a2fd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c01028e0 0x7fa3c01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c0100d50 0x7fa3c0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 --2- 172.21.15.86:0/455540659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0101f40 0x7fa3c0199440 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 >> 172.21.15.86:0/455540659 conn(0x7fa3c00fc2c0 msgr2=0x7fa3c0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 shutdown_connections 2024-03-22T19:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:24.963+0000 7fa3c7200700 1 -- 172.21.15.86:0/455540659 wait complete. 2024-03-22T19:11:24.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 -- 172.21.15.86:0/1544826701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c1028d0 msgr2=0x7fb13c109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 --2- 172.21.15.86:0/1544826701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c1028d0 0x7fb13c109180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b0a0 tx=0x7fb13400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 -- 172.21.15.86:0/1544826701 shutdown_connections 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 --2- 172.21.15.86:0/1544826701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c1028d0 0x7fb13c109180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 --2- 172.21.15.86:0/1544826701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb13c100d40 0x7fb13c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 --2- 172.21.15.86:0/1544826701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101f30 0x7fb13c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 -- 172.21.15.86:0/1544826701 >> 172.21.15.86:0/1544826701 conn(0x7fb13c0fc2b0 msgr2=0x7fb13c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 -- 172.21.15.86:0/1544826701 shutdown_connections 2024-03-22T19:11:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.175+0000 7fb142800700 1 -- 172.21.15.86:0/1544826701 wait complete. 2024-03-22T19:11:25.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.176+0000 7fb142800700 1 Processor -- start 2024-03-22T19:11:25.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.176+0000 7fb142800700 1 -- start start 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb142800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 0x7fb13c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb142800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c101f30 0x7fb13c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb142800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb13c1028d0 0x7fb13c1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb142800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c1a0af0 con 0x7fb13c100d40 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb142800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c1a0c30 con 0x7fb13c1028d0 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb142800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c1a0dc0 con 0x7fb13c101f30 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 0x7fb13c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 0x7fb13c198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39070/0 (socket says 172.21.15.86:39070) 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 -- 172.21.15.86:0/1836867340 learned_addr learned my addr 172.21.15.86:0/1836867340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13be00700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb13c1028d0 0x7fb13c1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13aa00700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c101f30 0x7fb13c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 -- 172.21.15.86:0/1836867340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c101f30 msgr2=0x7fb13c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c101f30 0x7fb13c199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 -- 172.21.15.86:0/1836867340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb13c1028d0 msgr2=0x7fb13c1a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb13c1028d0 0x7fb13c1a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.177+0000 7fb13b400700 1 -- 172.21.15.86:0/1836867340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134009d00 con 0x7fb13c100d40 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.178+0000 7fb13b400700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 0x7fb13c198fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00e970 tx=0x7fb12c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.178+0000 7fb133e00700 1 -- 172.21.15.86:0/1836867340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c009790 con 0x7fb13c100d40 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.178+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c1a10a0 con 0x7fb13c100d40 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.178+0000 7fb133e00700 1 -- 172.21.15.86:0/1836867340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12c00c7e0 con 0x7fb13c100d40 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.178+0000 7fb133e00700 1 -- 172.21.15.86:0/1836867340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c003c70 con 0x7fb13c100d40 2024-03-22T19:11:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.178+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c1a15b0 con 0x7fb13c100d40 2024-03-22T19:11:25.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.181+0000 7fb133e00700 1 -- 172.21.15.86:0/1836867340 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fb12c00cc50 con 0x7fb13c100d40 2024-03-22T19:11:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.182+0000 7fb133e00700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1200a0a90 0x7fb1200a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.182+0000 7fb1562a1b80 1 Processor -- start 2024-03-22T19:11:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.182+0000 7fb133e00700 1 -- 172.21.15.86:0/1836867340 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb12c014070 con 0x7fb13c100d40 2024-03-22T19:11:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.182+0000 7fb1562a1b80 1 -- start start 2024-03-22T19:11:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb13aa00700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1200a0a90 0x7fb1200a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 0x5593f86c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86c1420 0x5593f86c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f8618990 con 0x5593f86c0580 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f8616350 con 0x5593f86c1420 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f8611c40 con 0x5593f86bfc20 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb13b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 0x5593f86c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb13b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 0x5593f86c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51376/0 (socket says 172.21.15.86:51376) 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb13b400700 1 -- 172.21.15.86:0/17917699 learned_addr learned my addr 172.21.15.86:0/17917699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:25.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb13aa00700 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.183+0000 7fb13be00700 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86c1420 0x5593f86c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13b400700 1 -- 172.21.15.86:0/17917699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86c1420 msgr2=0x5593f86c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13aa00700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1200a0a90 0x7fb1200a2f50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb128009cb0 tx=0x7fb128011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13b400700 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86c1420 0x5593f86c18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13b400700 1 -- 172.21.15.86:0/17917699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 msgr2=0x5593f86c0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13b400700 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13b400700 1 -- 172.21.15.86:0/17917699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f84933b0 con 0x5593f86bfc20 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb13b400700 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 0x5593f86c0040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb12c099840 tx=0x7fb12c099290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74fb8fa10b38a7c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb130c00700 1 -- 172.21.15.86:0/17917699 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b1070 con 0x5593f86bfc20 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb130c00700 1 -- 172.21.15.86:0/17917699 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12c0b2270 con 0x5593f86bfc20 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.184+0000 7fb130c00700 1 -- 172.21.15.86:0/17917699 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b7ea0 con 0x5593f86bfc20 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 -- 172.21.15.86:0/17917699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 msgr2=0x5593f86c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 0x5593f86c0040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb12c099840 tx=0x7fb12c099290 comp rx=0 tx=0).stop 2024-03-22T19:11:25.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 -- 172.21.15.86:0/17917699 shutdown_connections 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86c1420 0x5593f86c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86bfc20 0x5593f86c0040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/17917699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 -- 172.21.15.86:0/17917699 >> 172.21.15.86:0/17917699 conn(0x5593f8635220 msgr2=0x5593f86babd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 -- 172.21.15.86:0/17917699 shutdown_connections 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.185+0000 7fb1562a1b80 1 -- 172.21.15.86:0/17917699 wait complete. 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 -- start start 2024-03-22T19:11:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 Processor -- start 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c0580 0x5593f86ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 0x5593f86d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f85add80 con 0x5593f86c1420 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f85ada20 con 0x5593f86bfc20 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f84956a0 con 0x5593f86c0580 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb1562a1b80 10 client.0 _ll_register_callbacks cb 0x5593f84193a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb13be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 0x5593f86d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.186+0000 7fb13be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 0x5593f86d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39080/0 (socket says 172.21.15.86:39080) 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 -- 172.21.15.86:0/2608501102 learned_addr learned my addr 172.21.15.86:0/2608501102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13b400700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13aa00700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c0580 0x5593f86ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 -- 172.21.15.86:0/2608501102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c0580 msgr2=0x5593f86ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c0580 0x5593f86ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 -- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 msgr2=0x5593f86c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 -- 172.21.15.86:0/2608501102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c1a15b0 con 0x5593f86c1420 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13b400700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb13be00700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 0x5593f86d1060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b0a0 tx=0x7fb134007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796cc25240e22825 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb125600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb125600700 1 -- 172.21.15.86:0/2608501102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400e040 con 0x5593f86c1420 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593f86b2ad0 con 0x5593f86c1420 2024-03-22T19:11:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.187+0000 7fb125600700 1 -- 172.21.15.86:0/2608501102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb134003a00 con 0x5593f86c1420 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb125600700 1 -- 172.21.15.86:0/2608501102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb134003bc0 con 0x5593f86c1420 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 10 client.15483 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5593f843fb60 con 0x5593f86c1420 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb125600700 1 -- 172.21.15.86:0/2608501102 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593f86b2ad0 con 0x5593f86c1420 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb125600700 1 -- 172.21.15.86:0/2608501102 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb134023020 con 0x5593f86c1420 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 20 client.15483 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f86d5940 0x5593f86d7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 4 client.15483 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593f860a700 con 0x5593f86d5940 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.188+0000 7fb13b400700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f86d5940 0x5593f86d7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.189+0000 7fb13b400700 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f86d5940 0x5593f86d7d60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3511b65367a5580 server_cookie=b9da7da58776ce2b in_seq=0 out_seq=0 2024-03-22T19:11:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.189+0000 7fb125600700 10 client.15483 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb125600700 1 -- 172.21.15.86:0/2608501102 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5593f860a700 con 0x5593f86d5940 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb125600700 10 client.15483 handle_command_reply: tid=0 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 client.15483 shutdown 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 msgr2=0x5593f86d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 0x5593f86d1060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b0a0 tx=0x7fb134007990 comp rx=0 tx=0).stop 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 shutdown_connections 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f86d5940 0x5593f86d7d60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c1420 0x5593f86d1060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2608501102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c0580 0x5593f86ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb125600700 0 client.15483 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 >> 172.21.15.86:0/2608501102 conn(0x5593f8635220 msgr2=0x5593f86ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 shutdown_connections 2024-03-22T19:11:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2608501102 wait complete. 2024-03-22T19:11:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.190+0000 7fb1562a1b80 20 client.15483 trim_cache size 0 max 16384 2024-03-22T19:11:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.203+0000 7fb1562a1b80 1 Processor -- start 2024-03-22T19:11:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.203+0000 7fb1562a1b80 1 -- start start 2024-03-22T19:11:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f86d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f86c9720 con 0x5593f86c0580 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f86dca80 con 0x5593f86bfc20 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f86b2610 con 0x5593f86c1420 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40174/0 (socket says 172.21.15.86:40174) 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13b400700 1 -- 172.21.15.86:0/2911973365 learned_addr learned my addr 172.21.15.86:0/2911973365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13be00700 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f86d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13b400700 1 -- 172.21.15.86:0/2911973365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 msgr2=0x5593f86d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13b400700 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f86d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.204+0000 7fb13b400700 1 -- 172.21.15.86:0/2911973365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 msgr2=0x5593f86c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb13b400700 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb13b400700 1 -- 172.21.15.86:0/2911973365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f85d7ea0 con 0x5593f86bfc20 2024-03-22T19:11:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb13aa00700 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb13b400700 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cef70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005950 tx=0x7fb12c099770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4fb068324e8a7e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb131600700 1 -- 172.21.15.86:0/2911973365 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b1040 con 0x5593f86bfc20 2024-03-22T19:11:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb131600700 1 -- 172.21.15.86:0/2911973365 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12c097c50 con 0x5593f86bfc20 2024-03-22T19:11:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.205+0000 7fb131600700 1 -- 172.21.15.86:0/2911973365 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c099000 con 0x5593f86bfc20 2024-03-22T19:11:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2911973365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 msgr2=0x5593f86cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cef70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005950 tx=0x7fb12c099770 comp rx=0 tx=0).stop 2024-03-22T19:11:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2911973365 shutdown_connections 2024-03-22T19:11:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f86d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cef70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/2911973365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f86c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2911973365 >> 172.21.15.86:0/2911973365 conn(0x5593f8635220 msgr2=0x5593f86b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2911973365 shutdown_connections 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 -- 172.21.15.86:0/2911973365 wait complete. 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 -- start start 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.206+0000 7fb1562a1b80 1 Processor -- start 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f8723430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f8725980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f8726090 con 0x5593f86c0580 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f8726200 con 0x5593f86bfc20 2024-03-22T19:11:25.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f87263c0 con 0x5593f86c1420 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb1562a1b80 10 client.0 _ll_register_callbacks cb 0x7fb1340074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f8723430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f8723430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39102/0 (socket says 172.21.15.86:39102) 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 -- 172.21.15.86:0/371449071 learned_addr learned my addr 172.21.15.86:0/371449071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13b400700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13be00700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f8725980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 -- 172.21.15.86:0/371449071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 msgr2=0x5593f8725980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f8725980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 -- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 msgr2=0x5593f86cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.207+0000 7fb13aa00700 1 -- 172.21.15.86:0/371449071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12800f520 con 0x5593f86c0580 2024-03-22T19:11:25.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb13aa00700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f8723430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb12800fd90 tx=0x7fb1280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3215f278356c355c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb125600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb125600700 1 -- 172.21.15.86:0/371449071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800fe80 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb125600700 1 -- 172.21.15.86:0/371449071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb1280283f0 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb12c0b2270 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb125600700 1 -- 172.21.15.86:0/371449071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1280285f0 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb1562a1b80 10 client.15489 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5593f84933b0 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb125600700 1 -- 172.21.15.86:0/371449071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb128037020 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.208+0000 7fb125600700 1 -- 172.21.15.86:0/371449071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb128027070 con 0x5593f86c0580 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb1562a1b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb1562a1b80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb1562a1b80 20 client.15489 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:25.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f8727ef0 0x5593f872a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb1562a1b80 4 client.15489 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593f8460cf0 con 0x5593f8727ef0 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb13b400700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f8727ef0 0x5593f872a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb13b400700 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f8727ef0 0x5593f872a310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d82b4b10a54041 server_cookie=473dbd6aafc78ff5 in_seq=0 out_seq=0 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.209+0000 7fb125600700 10 client.15489 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb125600700 1 -- 172.21.15.86:0/371449071 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593f8460cf0 con 0x5593f8727ef0 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb125600700 10 client.15489 handle_command_reply: tid=0 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 client.15489 shutdown 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 msgr2=0x5593f8723430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f8723430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb12800fd90 tx=0x7fb1280047c0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 shutdown_connections 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5593f8727ef0 0x5593f872a310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593f86c1420 0x5593f8725980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f86bfc20 0x5593f86cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 --2- 172.21.15.86:0/371449071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593f86c0580 0x5593f8723430 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb125600700 0 client.15489 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 >> 172.21.15.86:0/371449071 conn(0x5593f8635220 msgr2=0x5593f86d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 shutdown_connections 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 1 -- 172.21.15.86:0/371449071 wait complete. 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.210+0000 7fb1562a1b80 20 client.15489 trim_cache size 0 max 16384 2024-03-22T19:11:25.217 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1200a0a90 msgr2=0x7fb1200a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1200a0a90 0x7fb1200a2f50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb128009cb0 tx=0x7fb128011720 comp rx=0 tx=0).stop 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 msgr2=0x7fb13c198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 0x7fb13c198fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00e970 tx=0x7fb12c00ec80 comp rx=0 tx=0).stop 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 shutdown_connections 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb1200a0a90 0x7fb1200a2f50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb13c1028d0 0x7fb13c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c100d40 0x7fb13c198fc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 --2- 172.21.15.86:0/1836867340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c101f30 0x7fb13c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:25.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 >> 172.21.15.86:0/1836867340 conn(0x7fb13c0fc2b0 msgr2=0x7fb13c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:25.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 shutdown_connections 2024-03-22T19:11:25.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:25.211+0000 7fb142800700 1 -- 172.21.15.86:0/1836867340 wait complete. 2024-03-22T19:11:25.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:25 smithi086 ceph-mon[38032]: pgmap v299: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:25 smithi167 ceph-mon[47728]: pgmap v299: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:25 smithi131 ceph-mon[78559]: pgmap v299: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:26.231 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:11:26.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 -- 172.21.15.86:0/1340379107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8101f30 msgr2=0x7ff4d8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 --2- 172.21.15.86:0/1340379107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8101f30 0x7ff4d8102390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40099c0 tx=0x7ff4c4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 -- 172.21.15.86:0/1340379107 shutdown_connections 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 --2- 172.21.15.86:0/1340379107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d81028d0 0x7ff4d8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 --2- 172.21.15.86:0/1340379107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d8100d40 0x7ff4d8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 --2- 172.21.15.86:0/1340379107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8101f30 0x7ff4d8102390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 -- 172.21.15.86:0/1340379107 >> 172.21.15.86:0/1340379107 conn(0x7ff4d80fc2f0 msgr2=0x7ff4d80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 -- 172.21.15.86:0/1340379107 shutdown_connections 2024-03-22T19:11:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.437+0000 7ff4e0400700 1 -- 172.21.15.86:0/1340379107 wait complete. 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.438+0000 7ff4e0400700 1 Processor -- start 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.438+0000 7ff4e0400700 1 -- start start 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4e0400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d8100d40 0x7ff4d8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4e0400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8101f30 0x7ff4d8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4e0400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 0x7ff4d81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4e0400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81a0a20 con 0x7ff4d81028d0 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4e0400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81a0b60 con 0x7ff4d8101f30 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4e0400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81a0cf0 con 0x7ff4d8100d40 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 0x7ff4d81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 0x7ff4d81a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39118/0 (socket says 172.21.15.86:39118) 2024-03-22T19:11:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 -- 172.21.15.86:0/3214815819 learned_addr learned my addr 172.21.15.86:0/3214815819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4dda00700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d8100d40 0x7ff4d8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4dd000700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8101f30 0x7ff4d8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 -- 172.21.15.86:0/3214815819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d8100d40 msgr2=0x7ff4d8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d8100d40 0x7ff4d8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 -- 172.21.15.86:0/3214815819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8101f30 msgr2=0x7ff4d8199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8101f30 0x7ff4d8199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.439+0000 7ff4de400700 1 -- 172.21.15.86:0/3214815819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40095f0 con 0x7ff4d81028d0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.440+0000 7ff4de400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 0x7ff4d81a0450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ef30 tx=0x7ff4d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.440+0000 7ff4d6000700 1 -- 172.21.15.86:0/3214815819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0018070 con 0x7ff4d81028d0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.440+0000 7ff4d6000700 1 -- 172.21.15.86:0/3214815819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4d000f040 con 0x7ff4d81028d0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.440+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d81a0f70 con 0x7ff4d81028d0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.440+0000 7ff4d6000700 1 -- 172.21.15.86:0/3214815819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000cd00 con 0x7ff4d81028d0 2024-03-22T19:11:26.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.440+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d81a13f0 con 0x7ff4d81028d0 2024-03-22T19:11:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.443+0000 7ff4d6000700 1 -- 172.21.15.86:0/3214815819 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff4d0003a00 con 0x7ff4d81028d0 2024-03-22T19:11:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 Processor -- start 2024-03-22T19:11:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 -- start start 2024-03-22T19:11:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4d6000700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff4c00a0a90 0x7ff4c00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4d6000700 1 -- 172.21.15.86:0/3214815819 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff4d00cbb40 con 0x7ff4d81028d0 2024-03-22T19:11:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad7fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4dd000700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff4c00a0a90 0x7ff4c00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad80920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690acd7a10 con 0x55690ad7f600 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690acd53d0 con 0x55690ad7eca0 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.445+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690acd0cc0 con 0x55690ad804a0 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad80920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad80920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51446/0 (socket says 172.21.15.86:51446) 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 -- 172.21.15.86:0/1999791583 learned_addr learned my addr 172.21.15.86:0/1999791583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4dd000700 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad7fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4dda00700 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 -- 172.21.15.86:0/1999791583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 msgr2=0x55690ad7f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad7f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 -- 172.21.15.86:0/1999791583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 msgr2=0x55690ad7fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad7fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4de400700 1 -- 172.21.15.86:0/1999791583 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55690ab52430 con 0x55690ad804a0 2024-03-22T19:11:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.446+0000 7ff4dd000700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff4c00a0a90 0x7ff4c00a2f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c400ba30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.447+0000 7ff4de400700 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad80920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009c10 tx=0x7ff4d0098e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3841f3ed82e3c828 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.447+0000 7ff4caa00700 1 -- 172.21.15.86:0/1999791583 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001b320 con 0x55690ad804a0 2024-03-22T19:11:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.447+0000 7ff4caa00700 1 -- 172.21.15.86:0/1999791583 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4d0099950 con 0x55690ad804a0 2024-03-22T19:11:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.447+0000 7ff4caa00700 1 -- 172.21.15.86:0/1999791583 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00a1640 con 0x55690ad804a0 2024-03-22T19:11:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.447+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/1999791583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 msgr2=0x55690ad80920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.447+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad80920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009c10 tx=0x7ff4d0098e10 comp rx=0 tx=0).stop 2024-03-22T19:11:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/1999791583 shutdown_connections 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad80920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/1999791583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad7fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/1999791583 >> 172.21.15.86:0/1999791583 conn(0x55690acf42a0 msgr2=0x55690ad79c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/1999791583 shutdown_connections 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/1999791583 wait complete. 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 -- start start 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.448+0000 7ff4f3edfb80 1 Processor -- start 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ac6ce00 con 0x55690ad7f600 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ac6caa0 con 0x55690ad7eca0 2024-03-22T19:11:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ab54720 con 0x55690ad804a0 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4f3edfb80 10 client.0 _ll_register_callbacks cb 0x55690aad8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad890c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39144/0 (socket says 172.21.15.86:39144) 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 -- 172.21.15.86:0/4046760901 learned_addr learned my addr 172.21.15.86:0/4046760901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dda00700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4de400700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 -- 172.21.15.86:0/4046760901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 msgr2=0x55690ad900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad900b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 -- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 msgr2=0x55690ad88b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad88b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4dd000700 1 -- 172.21.15.86:0/4046760901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d81a13f0 con 0x55690ad7f600 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.449+0000 7ff4de400700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4dd000700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad890c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005f50 tx=0x7ff4c4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160473a0a474e621 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4bf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4bf400700 1 -- 172.21.15.86:0/4046760901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c401c030 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4bf400700 1 -- 172.21.15.86:0/4046760901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff4c400fde0 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55690ad71b50 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4bf400700 1 -- 172.21.15.86:0/4046760901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c4030430 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4f3edfb80 10 client.15501 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55690aafebe0 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.450+0000 7ff4bf400700 1 -- 172.21.15.86:0/4046760901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c403f020 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4bf400700 1 -- 172.21.15.86:0/4046760901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4c4039590 con 0x55690ad7f600 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4f3edfb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4f3edfb80 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:26.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4f3edfb80 20 client.15501 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ad948a0 0x55690ad96cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4f3edfb80 4 client.15501 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55690acc9780 con 0x55690ad948a0 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.451+0000 7ff4dda00700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ad948a0 0x55690ad96cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.452+0000 7ff4bf400700 10 client.15501 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.452+0000 7ff4dda00700 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ad948a0 0x55690ad96cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec13ea786efec04 server_cookie=9ca27f526e22718f in_seq=0 out_seq=0 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.452+0000 7ff4bf400700 1 -- 172.21.15.86:0/4046760901 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55690acc9780 con 0x55690ad948a0 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.452+0000 7ff4bf400700 10 client.15501 handle_command_reply: tid=0 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.452+0000 7ff4f3edfb80 1 client.15501 shutdown 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 msgr2=0x55690ad890c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad890c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005f50 tx=0x7ff4c4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 shutdown_connections 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ad948a0 0x55690ad96cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ad900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/4046760901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690ad890c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 >> 172.21.15.86:0/4046760901 conn(0x55690acf42a0 msgr2=0x55690ad79270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4bf400700 0 client.15501 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 shutdown_connections 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/4046760901 wait complete. 2024-03-22T19:11:26.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.453+0000 7ff4f3edfb80 20 client.15501 trim_cache size 0 max 16384 2024-03-22T19:11:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 Processor -- start 2024-03-22T19:11:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 -- start start 2024-03-22T19:11:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad8e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad7f600 0x55690ad968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad804a0 0x55690ad88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ad80d40 con 0x55690ad804a0 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ad886f0 con 0x55690ad7eca0 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.470+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ad76660 con 0x55690ad7f600 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad8e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4de400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad804a0 0x55690ad88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad8e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40222/0 (socket says 172.21.15.86:40222) 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 -- 172.21.15.86:0/862269199 learned_addr learned my addr 172.21.15.86:0/862269199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dd000700 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad7f600 0x55690ad968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 -- 172.21.15.86:0/862269199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad7f600 msgr2=0x55690ad968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad7f600 0x55690ad968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 -- 172.21.15.86:0/862269199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad804a0 msgr2=0x55690ad88b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad804a0 0x55690ad88b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 -- 172.21.15.86:0/862269199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55690ac96f20 con 0x55690ad7eca0 2024-03-22T19:11:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4de400700 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad804a0 0x55690ad88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4dda00700 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad8e060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8101d90 tx=0x7ff4cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dea89970a5b96d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.471+0000 7ff4cb400700 1 -- 172.21.15.86:0/862269199 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc017030 con 0x55690ad7eca0 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4cb400700 1 -- 172.21.15.86:0/862269199 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4cc018040 con 0x55690ad7eca0 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4cb400700 1 -- 172.21.15.86:0/862269199 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc003d90 con 0x55690ad7eca0 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/862269199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 msgr2=0x55690ad8e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad8e060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8101d90 tx=0x7ff4cc009260 comp rx=0 tx=0).stop 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/862269199 shutdown_connections 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad804a0 0x55690ad88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690ad8e060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/862269199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad7f600 0x55690ad968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/862269199 >> 172.21.15.86:0/862269199 conn(0x55690acf42a0 msgr2=0x55690ad79b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/862269199 shutdown_connections 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.472+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/862269199 wait complete. 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 -- start start 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 Processor -- start 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690adde870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690addedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ade5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ade6330 con 0x55690ad7f600 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ade6470 con 0x55690ad7eca0 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690ade6600 con 0x55690ad804a0 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4f3edfb80 10 client.0 _ll_register_callbacks cb 0x7ff4c400b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690addedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690adde870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690addedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39154/0 (socket says 172.21.15.86:39154) 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690adde870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40238/0 (socket says 172.21.15.86:40238) 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.473+0000 7ff4dd000700 1 -- 172.21.15.86:0/849579522 learned_addr learned my addr 172.21.15.86:0/849579522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4de400700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ade5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dd000700 1 -- 172.21.15.86:0/849579522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 msgr2=0x55690ade5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dd000700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ade5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dd000700 1 -- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 msgr2=0x55690adde870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dd000700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690adde870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dd000700 1 -- 172.21.15.86:0/849579522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55690ab52430 con 0x55690ad7f600 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dda00700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690adde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4dd000700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690addedb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4039940 tx=0x7ff4c4039e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c093d6d8d1b03a1b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4bf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4bf400700 1 -- 172.21.15.86:0/849579522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c401c040 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4d0099950 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.474+0000 7ff4bf400700 1 -- 172.21.15.86:0/849579522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff4c4040080 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4bf400700 1 -- 172.21.15.86:0/849579522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c4030640 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 10 client.15507 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff4cc004040 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4bf400700 1 -- 172.21.15.86:0/849579522 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4d0099950 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4bf400700 1 -- 172.21.15.86:0/849579522 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4c4044630 con 0x55690ad7f600 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 20 client.15507 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:26.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ade7c30 0x55690adea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 4 client.15507 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:11:26.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55690ab1fd70 con 0x55690ade7c30 2024-03-22T19:11:26.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.475+0000 7ff4dda00700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ade7c30 0x55690adea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.476+0000 7ff4dda00700 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ade7c30 0x55690adea050 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130dba9888744ec7 server_cookie=f0205b35f80ac80e in_seq=0 out_seq=0 2024-03-22T19:11:26.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.476+0000 7ff4bf400700 10 client.15507 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4bf400700 1 -- 172.21.15.86:0/849579522 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55690ab1fd70 con 0x55690ade7c30 2024-03-22T19:11:26.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4bf400700 10 client.15507 handle_command_reply: tid=0 2024-03-22T19:11:26.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 client.15507 shutdown 2024-03-22T19:11:26.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 msgr2=0x55690addedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690addedb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4039940 tx=0x7ff4c4039e10 comp rx=0 tx=0).stop 2024-03-22T19:11:26.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 shutdown_connections 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55690ade7c30 0x55690adea050 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55690ad804a0 0x55690ade5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55690ad7eca0 0x55690adde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 --2- 172.21.15.86:0/849579522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55690ad7f600 0x55690addedb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 >> 172.21.15.86:0/849579522 conn(0x55690acf42a0 msgr2=0x55690ad968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4bf400700 0 client.15507 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 shutdown_connections 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.477+0000 7ff4f3edfb80 1 -- 172.21.15.86:0/849579522 wait complete. 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4f3edfb80 20 client.15507 trim_cache size 0 max 16384 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:11:26.484 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "4e4fe75a-ac7a-4f4c-b92c-b217596613a5", 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff4c00a0a90 msgr2=0x7ff4c00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff4c00a0a90 0x7ff4c00a2f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c400ba30 comp rx=0 tx=0).stop 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 msgr2=0x7ff4d81a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 0x7ff4d81a0450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ef30 tx=0x7ff4d000c3a0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 shutdown_connections 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff4c00a0a90 0x7ff4c00a2f50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d81028d0 0x7ff4d81a0450 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d8100d40 0x7ff4d8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 --2- 172.21.15.86:0/3214815819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8101f30 0x7ff4d8199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 >> 172.21.15.86:0/3214815819 conn(0x7ff4d80fc2f0 msgr2=0x7ff4d8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 shutdown_connections 2024-03-22T19:11:26.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.478+0000 7ff4e0400700 1 -- 172.21.15.86:0/3214815819 wait complete. 2024-03-22T19:11:26.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 -- 172.21.15.86:0/3247417602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb081028e0 msgr2=0x7fbb08109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 --2- 172.21.15.86:0/3247417602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb081028e0 0x7fbb08109190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b0d0 tx=0x7fbb0000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 -- 172.21.15.86:0/3247417602 shutdown_connections 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 --2- 172.21.15.86:0/3247417602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb081028e0 0x7fbb08109190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 --2- 172.21.15.86:0/3247417602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb08100d50 0x7fbb08101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 --2- 172.21.15.86:0/3247417602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb08101f40 0x7fbb081023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 -- 172.21.15.86:0/3247417602 >> 172.21.15.86:0/3247417602 conn(0x7fbb080fc2c0 msgr2=0x7fbb080fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 -- 172.21.15.86:0/3247417602 shutdown_connections 2024-03-22T19:11:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.691+0000 7fbb0e400700 1 -- 172.21.15.86:0/3247417602 wait complete. 2024-03-22T19:11:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 Processor -- start 2024-03-22T19:11:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 -- start start 2024-03-22T19:11:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb08100d50 0x7fbb08074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb08101f40 0x7fbb080731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb081028e0 0x7fbb080736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb08073e80 con 0x7fbb08101f40 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb08073ff0 con 0x7fbb081028e0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.692+0000 7fbb0e400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081a0e30 con 0x7fbb08100d50 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb08101f40 0x7fbb080731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb08101f40 0x7fbb080731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39168/0 (socket says 172.21.15.86:39168) 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb06a00700 1 -- 172.21.15.86:0/902818069 learned_addr learned my addr 172.21.15.86:0/902818069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb081028e0 0x7fbb080736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb08100d50 0x7fbb08074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 -- 172.21.15.86:0/902818069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb08100d50 msgr2=0x7fbb08074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb08100d50 0x7fbb08074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 -- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb08101f40 msgr2=0x7fbb080731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb08101f40 0x7fbb080731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 -- 172.21.15.86:0/902818069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009d00 con 0x7fbb081028e0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.693+0000 7fbb07e00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb081028e0 0x7fbb080736e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbb00007b60 tx=0x7fbb00008e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.694+0000 7fbaffe00700 1 -- 172.21.15.86:0/902818069 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000e040 con 0x7fbb081028e0 2024-03-22T19:11:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.694+0000 7fbaffe00700 1 -- 172.21.15.86:0/902818069 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb00005070 con 0x7fbb081028e0 2024-03-22T19:11:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.694+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb081a1060 con 0x7fbb081028e0 2024-03-22T19:11:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.694+0000 7fbaffe00700 1 -- 172.21.15.86:0/902818069 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0001d6d0 con 0x7fbb081028e0 2024-03-22T19:11:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.694+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb081a1430 con 0x7fbb081028e0 2024-03-22T19:11:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.697+0000 7fbaffe00700 1 -- 172.21.15.86:0/902818069 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fbb000051e0 con 0x7fbb081028e0 2024-03-22T19:11:26.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 Processor -- start 2024-03-22T19:11:26.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 -- start start 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c60c990 con 0x55635c6b3c20 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c60a350 con 0x55635c6b5420 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c605c40 con 0x55635c6b4580 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb07400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39170/0 (socket says 172.21.15.86:39170) 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbb06a00700 1 -- 172.21.15.86:0/3753152002 learned_addr learned my addr 172.21.15.86:0/3753152002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.698+0000 7fbaffe00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbaec0a0a90 0x7fbaec0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbaffe00700 1 -- 172.21.15.86:0/902818069 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fbb000ccfa0 con 0x7fbb081028e0 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07400700 1 -- 172.21.15.86:0/3753152002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 msgr2=0x55635c6b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07400700 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07400700 1 -- 172.21.15.86:0/3753152002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 msgr2=0x55635c6b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07400700 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07400700 1 -- 172.21.15.86:0/3753152002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55635c4873b0 con 0x55635c6b4580 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07e00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbaec0a0a90 0x7fbaec0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbb07400700 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6b4e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009c80 tx=0x7fbaf8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ead0f4e4cdcef7e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbafcc00700 1 -- 172.21.15.86:0/3753152002 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8011cb0 con 0x55635c6b4580 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbafcc00700 1 -- 172.21.15.86:0/3753152002 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf8011e10 con 0x55635c6b4580 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.699+0000 7fbafcc00700 1 -- 172.21.15.86:0/3753152002 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8005220 con 0x55635c6b4580 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3753152002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 msgr2=0x55635c6b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb07e00700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbaec0a0a90 0x7fbaec0a2f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005040 tx=0x7fbb0000b3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6b4e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009c80 tx=0x7fbaf8009f90 comp rx=0 tx=0).stop 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3753152002 shutdown_connections 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3753152002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6b4e40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3753152002 >> 172.21.15.86:0/3753152002 conn(0x55635c629220 msgr2=0x55635c6aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3753152002 shutdown_connections 2024-03-22T19:11:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.700+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3753152002 wait complete. 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 -- start start 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 Processor -- start 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6bda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c5a1d80 con 0x55635c6b3c20 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c5a1a20 con 0x55635c6b5420 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.701+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c4896a0 con 0x55635c6b4580 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb21eb0b80 10 client.0 _ll_register_callbacks cb 0x55635c40d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6bda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6bda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39178/0 (socket says 172.21.15.86:39178) 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 -- 172.21.15.86:0/926326341 learned_addr learned my addr 172.21.15.86:0/926326341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb07400700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 -- 172.21.15.86:0/926326341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 msgr2=0x55635c6bdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6bdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 -- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 msgr2=0x55635c6c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 -- 172.21.15.86:0/926326341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf80098e0 con 0x55635c6b3c20 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbb06a00700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6bda40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4014b70 tx=0x7fbaf4007480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b101b01307b075a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.702+0000 7fbaf1600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbaf1600700 1 -- 172.21.15.86:0/926326341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017040 con 0x55635c6b3c20 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbaf1600700 1 -- 172.21.15.86:0/926326341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fbaf4014170 con 0x55635c6b3c20 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55635c6a6ad0 con 0x55635c6b3c20 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbaf1600700 1 -- 172.21.15.86:0/926326341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf4014330 con 0x55635c6b3c20 2024-03-22T19:11:26.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 10 client.15519 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55635c433b60 con 0x55635c6b3c20 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbaf1600700 1 -- 172.21.15.86:0/926326341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55635c6a6ad0 con 0x55635c6b3c20 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbaf1600700 1 -- 172.21.15.86:0/926326341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbaf4003710 con 0x55635c6b3c20 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 20 client.15519 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c6c9770 0x55635c6cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 4 client.15519 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.703+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55635c5fe700 con 0x55635c6c9770 2024-03-22T19:11:26.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.704+0000 7fbb07400700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c6c9770 0x55635c6cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.704+0000 7fbb07400700 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c6c9770 0x55635c6cbb90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915b95cf6224dd20 server_cookie=55f7d1d1aa027b7c in_seq=0 out_seq=0 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.704+0000 7fbaf1600700 10 client.15519 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbaf1600700 1 -- 172.21.15.86:0/926326341 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55635c5fe700 con 0x55635c6c9770 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbaf1600700 10 client.15519 handle_command_reply: tid=0 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 client.15519 shutdown 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 msgr2=0x55635c6bda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6bda40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4014b70 tx=0x7fbaf4007480 comp rx=0 tx=0).stop 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 shutdown_connections 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c6c9770 0x55635c6cbb90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6bda40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/926326341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b4580 0x55635c6bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 >> 172.21.15.86:0/926326341 conn(0x55635c629220 msgr2=0x55635c6ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbaf1600700 0 client.15519 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 shutdown_connections 2024-03-22T19:11:26.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/926326341 wait complete. 2024-03-22T19:11:26.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.705+0000 7fbb21eb0b80 20 client.15519 trim_cache size 0 max 16384 2024-03-22T19:11:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.722+0000 7fbb21eb0b80 1 Processor -- start 2024-03-22T19:11:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.722+0000 7fbb21eb0b80 1 -- start start 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 0x55635c6c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b5420 0x55635c6b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c6ab5e0 con 0x55635c6b3c20 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c6c6cc0 con 0x55635c6b4580 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c6b03b0 con 0x55635c6b5420 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 0x55635c6c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 0x55635c6c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40310/0 (socket says 172.21.15.86:40310) 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 -- 172.21.15.86:0/3802284527 learned_addr learned my addr 172.21.15.86:0/3802284527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b5420 0x55635c6b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 -- 172.21.15.86:0/3802284527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b5420 msgr2=0x55635c6b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b5420 0x55635c6b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 -- 172.21.15.86:0/3802284527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 msgr2=0x55635c6b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 -- 172.21.15.86:0/3802284527 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55635c5cbea0 con 0x55635c6b4580 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b5420 0x55635c6b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.723+0000 7fbb07400700 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 0x55635c6c2f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009c80 tx=0x7fbaf8009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e88c5a210bec45 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbafd600700 1 -- 172.21.15.86:0/3802284527 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8011c30 con 0x55635c6b4580 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbafd600700 1 -- 172.21.15.86:0/3802284527 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf8005ce0 con 0x55635c6b4580 2024-03-22T19:11:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbafd600700 1 -- 172.21.15.86:0/3802284527 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8003cd0 con 0x55635c6b4580 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3802284527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 msgr2=0x55635c6c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 0x55635c6c2f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009c80 tx=0x7fbaf8009bf0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3802284527 shutdown_connections 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b5420 0x55635c6b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b3c20 0x55635c6b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3802284527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b4580 0x55635c6c2f70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3802284527 >> 172.21.15.86:0/3802284527 conn(0x55635c629220 msgr2=0x55635c6aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.724+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3802284527 shutdown_connections 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.725+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3802284527 wait complete. 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.725+0000 7fbb21eb0b80 1 -- start start 2024-03-22T19:11:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.725+0000 7fbb21eb0b80 1 Processor -- start 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b3c20 0x55635c6c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 0x55635c6c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c6c1530 con 0x55635c6b4580 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c6c16a0 con 0x55635c6b5420 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635c6c1860 con 0x55635c6b3c20 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb21eb0b80 10 client.0 _ll_register_callbacks cb 0x7fbaf40145d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 0x55635c6c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 0x55635c6c2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39192/0 (socket says 172.21.15.86:39192) 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 -- 172.21.15.86:0/3938932877 learned_addr learned my addr 172.21.15.86:0/3938932877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b3c20 0x55635c6c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 -- 172.21.15.86:0/3938932877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b3c20 msgr2=0x55635c6c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b3c20 0x55635c6c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 -- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 msgr2=0x55635c6c0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb07400700 1 -- 172.21.15.86:0/3938932877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf801d040 con 0x55635c6b4580 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.726+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b3c20 0x55635c6c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbb07400700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 0x55635c6c2cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8004720 tx=0x7fbaf80096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f1fc2e82026deb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbaf1600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbaf1600700 1 -- 172.21.15.86:0/3938932877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8004050 con 0x55635c6b4580 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbaf1600700 1 -- 172.21.15.86:0/3938932877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fbaf80041b0 con 0x55635c6b4580 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbaf8011e10 con 0x55635c6b4580 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbaf1600700 1 -- 172.21.15.86:0/3938932877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf8004390 con 0x55635c6b4580 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbb21eb0b80 10 client.15525 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55635c48bcd0 con 0x55635c6b4580 2024-03-22T19:11:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbaf1600700 1 -- 172.21.15.86:0/3938932877 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf8011e10 con 0x55635c6b4580 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.727+0000 7fbaf1600700 1 -- 172.21.15.86:0/3938932877 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbaf80268b0 con 0x55635c6b4580 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb21eb0b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb21eb0b80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb21eb0b80 20 client.15525 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c71c0f0 0x55635c71e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb21eb0b80 4 client.15525 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55635c454cf0 con 0x55635c71c0f0 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c71c0f0 0x55635c71e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbb06a00700 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c71c0f0 0x55635c71e510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8762ed1ec1f4909 server_cookie=1f8c8677076e670f in_seq=0 out_seq=0 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbaf1600700 10 client.15525 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.728+0000 7fbaf1600700 1 -- 172.21.15.86:0/3938932877 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55635c454cf0 con 0x55635c71c0f0 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbaf1600700 10 client.15525 handle_command_reply: tid=0 2024-03-22T19:11:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 client.15525 shutdown 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 msgr2=0x55635c6c2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 0x55635c6c2cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8004720 tx=0x7fbaf80096f0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 shutdown_connections 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55635c71c0f0 0x55635c71e510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55635c6b5420 0x55635c6c0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55635c6b3c20 0x55635c6c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 --2- 172.21.15.86:0/3938932877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635c6b4580 0x55635c6c2cd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbaf1600700 0 client.15525 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 >> 172.21.15.86:0/3938932877 conn(0x55635c629220 msgr2=0x55635c6bb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 shutdown_connections 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 1 -- 172.21.15.86:0/3938932877 wait complete. 2024-03-22T19:11:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb21eb0b80 20 client.15525 trim_cache size 0 max 16384 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: "4e4fe75a-ac7a-4f4c-b92c-b217596613a5": { 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "4e4fe75a-ac7a-4f4c-b92c-b217596613a5", 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:11:26.736 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbaec0a0a90 msgr2=0x7fbaec0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb0e400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbaec0a0a90 0x7fbaec0a2f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005040 tx=0x7fbb0000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb081028e0 msgr2=0x7fbb080736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.729+0000 7fbb0e400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb081028e0 0x7fbb080736e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbb00007b60 tx=0x7fbb00008e90 comp rx=0 tx=0).stop 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 shutdown_connections 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fbaec0a0a90 0x7fbaec0a2f50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb081028e0 0x7fbb080736e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb08100d50 0x7fbb08074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 --2- 172.21.15.86:0/902818069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb08101f40 0x7fbb080731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 >> 172.21.15.86:0/902818069 conn(0x7fbb080fc2c0 msgr2=0x7fbb08105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 shutdown_connections 2024-03-22T19:11:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:26.730+0000 7fbb0e400700 1 -- 172.21.15.86:0/902818069 wait complete. 2024-03-22T19:11:26.749 INFO:tasks.cephfs.filesystem:scrub status for tag:4e4fe75a-ac7a-4f4c-b92c-b217596613a5 - {'path': '/', 'tag': '4e4fe75a-ac7a-4f4c-b92c-b217596613a5', 'options': 'recursive,force'} 2024-03-22T19:11:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:27 smithi131 ceph-mon[78559]: pgmap v300: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:27 smithi086 ceph-mon[38032]: pgmap v300: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:27 smithi167 ceph-mon[47728]: pgmap v300: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:28.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:28 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:11:28.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:28 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:11:28.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:28 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:11:28.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:28 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:11:28.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:28 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:11:28.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:28 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:11:28.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:28 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:11:28.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:28 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:11:28.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:28 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:11:28.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:28 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:11:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:28 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:11:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:28 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:11:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:28 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:11:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:28 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:11:28.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:28 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:11:29.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:29 smithi131 ceph-mon[78559]: pgmap v301: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-22T19:11:29.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:29 smithi086 ceph-mon[38032]: pgmap v301: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-22T19:11:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:29 smithi167 ceph-mon[47728]: pgmap v301: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-22T19:11:31.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:31 smithi086 ceph-mon[38032]: pgmap v302: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-22T19:11:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:31 smithi167 ceph-mon[47728]: pgmap v302: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-22T19:11:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:31 smithi131 ceph-mon[78559]: pgmap v302: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-22T19:11:33.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:33 smithi086 ceph-mon[38032]: pgmap v303: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:33 smithi167 ceph-mon[47728]: pgmap v303: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:33 smithi131 ceph-mon[78559]: pgmap v303: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:35.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:35 smithi086 ceph-mon[38032]: pgmap v304: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:35 smithi167 ceph-mon[47728]: pgmap v304: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:35 smithi131 ceph-mon[78559]: pgmap v304: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:37.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:37 smithi131 ceph-mon[78559]: pgmap v305: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:37 smithi086 ceph-mon[38032]: pgmap v305: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:37 smithi167 ceph-mon[47728]: pgmap v305: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-22T19:11:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:39 smithi086 ceph-mon[38032]: pgmap v306: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-03-22T19:11:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:39 smithi167 ceph-mon[47728]: pgmap v306: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-03-22T19:11:39.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:39 smithi131 ceph-mon[78559]: pgmap v306: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-03-22T19:11:41.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:41 smithi086 ceph-mon[38032]: pgmap v307: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:11:41.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:41 smithi167 ceph-mon[47728]: pgmap v307: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:11:41.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:41 smithi131 ceph-mon[78559]: pgmap v307: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-22T19:11:43.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:43 smithi086 ceph-mon[38032]: pgmap v308: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-22T19:11:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:43 smithi167 ceph-mon[47728]: pgmap v308: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-22T19:11:43.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:43 smithi131 ceph-mon[78559]: pgmap v308: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-22T19:11:45.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:45 smithi086 ceph-mon[38032]: pgmap v309: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-03-22T19:11:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:45 smithi167 ceph-mon[47728]: pgmap v309: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-03-22T19:11:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:45 smithi131 ceph-mon[78559]: pgmap v309: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-03-22T19:11:47.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:47 smithi131 ceph-mon[78559]: pgmap v310: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:47.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:47 smithi086 ceph-mon[38032]: pgmap v310: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:47 smithi167 ceph-mon[47728]: pgmap v310: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:49.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:49 smithi086 ceph-mon[38032]: pgmap v311: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:49 smithi167 ceph-mon[47728]: pgmap v311: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:49.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:49 smithi131 ceph-mon[78559]: pgmap v311: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:51.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:51 smithi086 ceph-mon[38032]: pgmap v312: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:51 smithi167 ceph-mon[47728]: pgmap v312: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:51 smithi131 ceph-mon[78559]: pgmap v312: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:53.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:53 smithi086 ceph-mon[38032]: pgmap v313: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:53 smithi167 ceph-mon[47728]: pgmap v313: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:53 smithi131 ceph-mon[78559]: pgmap v313: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:11:55.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:55 smithi086 ceph-mon[38032]: pgmap v314: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:55 smithi167 ceph-mon[47728]: pgmap v314: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:55 smithi131 ceph-mon[78559]: pgmap v314: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-22T19:11:56.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:11:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 -- 172.21.15.86:0/1881637834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda001008c0 msgr2=0x7fda00100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 --2- 172.21.15.86:0/1881637834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda001008c0 0x7fda00100d40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b0a0 tx=0x7fd9f800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 -- 172.21.15.86:0/1881637834 shutdown_connections 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 --2- 172.21.15.86:0/1881637834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda001008c0 0x7fda00100d40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 --2- 172.21.15.86:0/1881637834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda000ff5a0 0x7fda000ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 --2- 172.21.15.86:0/1881637834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 0x7fda00100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 -- 172.21.15.86:0/1881637834 >> 172.21.15.86:0/1881637834 conn(0x7fda000fb1a0 msgr2=0x7fda000fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 -- 172.21.15.86:0/1881637834 shutdown_connections 2024-03-22T19:11:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.974+0000 7fda07400700 1 -- 172.21.15.86:0/1881637834 wait complete. 2024-03-22T19:11:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 Processor -- start 2024-03-22T19:11:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 -- start start 2024-03-22T19:11:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 0x7fda00198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 0x7fda001994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda001008c0 0x7fda001a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001a0ad0 con 0x7fda000ff5a0 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001a0c10 con 0x7fda000fff00 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.976+0000 7fda07400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001a0da0 con 0x7fda001008c0 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 0x7fda00198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 0x7fda00198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54848/0 (socket says 172.21.15.86:54848) 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 -- 172.21.15.86:0/650094421 learned_addr learned my addr 172.21.15.86:0/650094421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda05400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda001008c0 0x7fda001a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 0x7fda001994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 -- 172.21.15.86:0/650094421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda001008c0 msgr2=0x7fda001a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda001008c0 0x7fda001a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 -- 172.21.15.86:0/650094421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 msgr2=0x7fda001994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 0x7fda001994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.977+0000 7fda04a00700 1 -- 172.21.15.86:0/650094421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009d00 con 0x7fda000ff5a0 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 0x7fda001994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fda04a00700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 0x7fda00198fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400e9a0 tx=0x7fd9f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fd9fd600700 1 -- 172.21.15.86:0/650094421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4009790 con 0x7fda000ff5a0 2024-03-22T19:11:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fd9fd600700 1 -- 172.21.15.86:0/650094421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f400cbd0 con 0x7fda000ff5a0 2024-03-22T19:11:56.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fd9fd600700 1 -- 172.21.15.86:0/650094421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4018680 con 0x7fda000ff5a0 2024-03-22T19:11:56.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda001a1080 con 0x7fda000ff5a0 2024-03-22T19:11:56.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.978+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001a1540 con 0x7fda000ff5a0 2024-03-22T19:11:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.983+0000 7fd9fd600700 1 -- 172.21.15.86:0/650094421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fd9f400cd40 con 0x7fda000ff5a0 2024-03-22T19:11:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.985+0000 7fda1aec2b80 1 Processor -- start 2024-03-22T19:11:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.985+0000 7fd9fd600700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd9e80a0a90 0x7fd9e80a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.985+0000 7fda1aec2b80 1 -- start start 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.985+0000 7fd9fd600700 1 -- 172.21.15.86:0/650094421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fd9f4014070 con 0x7fda000ff5a0 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 0x55a9dfd4e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda05400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd9e80a0a90 0x7fd9e80a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd4f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfc9e990 con 0x55a9dfd4fc30 2024-03-22T19:11:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfc9c350 con 0x55a9dfd4e430 2024-03-22T19:11:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfc97c40 con 0x55a9dfd4ed90 2024-03-22T19:11:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda04a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 0x55a9dfd4e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda04a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 0x55a9dfd4e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44884/0 (socket says 172.21.15.86:44884) 2024-03-22T19:11:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda04a00700 1 -- 172.21.15.86:0/239956668 learned_addr learned my addr 172.21.15.86:0/239956668 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd4f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.986+0000 7fda05400700 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda04a00700 1 -- 172.21.15.86:0/239956668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4ed90 msgr2=0x55a9dfd4f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda04a00700 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd4f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda04a00700 1 -- 172.21.15.86:0/239956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4fc30 msgr2=0x55a9dfd500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda04a00700 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda04a00700 1 -- 172.21.15.86:0/239956668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9dfb193b0 con 0x55a9dfd4e430 2024-03-22T19:11:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda05400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd9e80a0a90 0x7fd9e80a2f50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009fd0 tx=0x7fd9f8007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fda04a00700 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 0x55a9dfd4e850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40963e0 tx=0x7fd9f40992d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1126ae79c1cd8a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fd9f1c00700 1 -- 172.21.15.86:0/239956668 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4099970 con 0x55a9dfd4e430 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fd9f1c00700 1 -- 172.21.15.86:0/239956668 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f409d7a0 con 0x55a9dfd4e430 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.987+0000 7fd9f1c00700 1 -- 172.21.15.86:0/239956668 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40990c0 con 0x55a9dfd4e430 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 -- 172.21.15.86:0/239956668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 msgr2=0x55a9dfd4e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 0x55a9dfd4e850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40963e0 tx=0x7fd9f40992d0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 -- 172.21.15.86:0/239956668 shutdown_connections 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4e430 0x55a9dfd4e850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/239956668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd4f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.988+0000 7fda1aec2b80 1 -- 172.21.15.86:0/239956668 >> 172.21.15.86:0/239956668 conn(0x55a9dfcbb220 msgr2=0x55a9dfd40bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.989+0000 7fda1aec2b80 1 -- 172.21.15.86:0/239956668 shutdown_connections 2024-03-22T19:11:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.989+0000 7fda1aec2b80 1 -- 172.21.15.86:0/239956668 wait complete. 2024-03-22T19:11:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.989+0000 7fda1aec2b80 1 -- start start 2024-03-22T19:11:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 Processor -- start 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 0x55a9dfd58300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd58840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfc33d80 con 0x55a9dfd4e430 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfc33a20 con 0x55a9dfd4ed90 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.990+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfb1b6a0 con 0x55a9dfd4fc30 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda1aec2b80 10 client.0 _ll_register_callbacks cb 0x55a9dfa9f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 0x55a9dfd58300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 0x55a9dfd58300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54866/0 (socket says 172.21.15.86:54866) 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 -- 172.21.15.86:0/1415105845 learned_addr learned my addr 172.21.15.86:0/1415105845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd58840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda05400700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 -- 172.21.15.86:0/1415105845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 msgr2=0x55a9dfd5f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd5f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 -- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 msgr2=0x55a9dfd58840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.991+0000 7fda04a00700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd58840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fda04a00700 1 -- 172.21.15.86:0/1415105845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001a1540 con 0x55a9dfd4e430 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fda04a00700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 0x55a9dfd58300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c5b0 tx=0x7fd9f409a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f38f027c95a974d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fd9e6000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fd9e6000700 1 -- 172.21.15.86:0/1415105845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40a1df0 con 0x55a9dfd4e430 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fd9e6000700 1 -- 172.21.15.86:0/1415105845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fd9f4099970 con 0x55a9dfd4e430 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9dfd38ad0 con 0x55a9dfd4e430 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fd9e6000700 1 -- 172.21.15.86:0/1415105845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f40a1960 con 0x55a9dfd4e430 2024-03-22T19:11:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fda1aec2b80 10 client.15537 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.993+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a9dfac5b60 con 0x55a9dfd4e430 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fd9e6000700 1 -- 172.21.15.86:0/1415105845 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9dfd38ad0 con 0x55a9dfd4e430 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fd9e6000700 1 -- 172.21.15.86:0/1415105845 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd9f405fc50 con 0x55a9dfd4e430 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fda1aec2b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fda1aec2b80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fda1aec2b80 20 client.15537 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfd63c00 0x55a9dfd65ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fda1aec2b80 4 client.15537 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.994+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9dfc90700 con 0x55a9dfd63c00 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.995+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfd63c00 0x55a9dfd65ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.995+0000 7fd9e6000700 10 client.15537 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.995+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfd63c00 0x55a9dfd65ff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf132e954ca4c5 server_cookie=6742f9631efb71c8 in_seq=0 out_seq=0 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fd9e6000700 1 -- 172.21.15.86:0/1415105845 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a9dfc90700 con 0x55a9dfd63c00 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fd9e6000700 10 client.15537 handle_command_reply: tid=0 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 client.15537 shutdown 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 msgr2=0x55a9dfd58300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 0x55a9dfd58300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c5b0 tx=0x7fd9f409a1b0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 shutdown_connections 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfd63c00 0x55a9dfd65ff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd5f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4e430 0x55a9dfd58300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/1415105845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd58840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 >> 172.21.15.86:0/1415105845 conn(0x55a9dfcbb220 msgr2=0x55a9dfd401f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.996+0000 7fd9e6000700 0 client.15537 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:57.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.997+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 shutdown_connections 2024-03-22T19:11:57.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.997+0000 7fda1aec2b80 1 -- 172.21.15.86:0/1415105845 wait complete. 2024-03-22T19:11:57.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:56.997+0000 7fda1aec2b80 20 client.15537 trim_cache size 0 max 16384 2024-03-22T19:11:57.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.014+0000 7fda1aec2b80 1 Processor -- start 2024-03-22T19:11:57.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.014+0000 7fda1aec2b80 1 -- start start 2024-03-22T19:11:57.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd5d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd50010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd56160 0x55a9dfd40b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfd6ad10 con 0x55a9dfd4ed90 2024-03-22T19:11:57.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfd41930 con 0x55a9dfd56160 2024-03-22T19:11:57.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfd38610 con 0x55a9dfd4fc30 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fd9ffe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd50010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda04a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd5d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fd9ffe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd50010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54686/0 (socket says 172.21.15.86:54686) 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fda05400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd56160 0x55a9dfd40b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.015+0000 7fd9ffe00700 1 -- 172.21.15.86:0/2336809304 learned_addr learned my addr 172.21.15.86:0/2336809304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda05400700 1 -- 172.21.15.86:0/2336809304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 msgr2=0x55a9dfd50010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda05400700 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd50010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda05400700 1 -- 172.21.15.86:0/2336809304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4ed90 msgr2=0x55a9dfd5d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda05400700 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd5d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda05400700 1 -- 172.21.15.86:0/2336809304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9dfc5dea0 con 0x55a9dfd56160 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda04a00700 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd5d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fda05400700 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd56160 0x55a9dfd40b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8007a60 tx=0x7fd9f8023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d3c360ca31daa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fd9f2600700 1 -- 172.21.15.86:0/2336809304 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800e040 con 0x55a9dfd56160 2024-03-22T19:11:57.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fd9f2600700 1 -- 172.21.15.86:0/2336809304 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f8003e60 con 0x55a9dfd56160 2024-03-22T19:11:57.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.016+0000 7fd9f2600700 1 -- 172.21.15.86:0/2336809304 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8005280 con 0x55a9dfd56160 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.017+0000 7fda1aec2b80 1 -- 172.21.15.86:0/2336809304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd56160 msgr2=0x55a9dfd40b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.017+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd56160 0x55a9dfd40b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8007a60 tx=0x7fd9f8023000 comp rx=0 tx=0).stop 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 -- 172.21.15.86:0/2336809304 shutdown_connections 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd56160 0x55a9dfd40b20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd4ed90 0x55a9dfd5d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/2336809304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfd50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 -- 172.21.15.86:0/2336809304 >> 172.21.15.86:0/2336809304 conn(0x55a9dfcbb220 msgr2=0x55a9dfd57840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 -- 172.21.15.86:0/2336809304 shutdown_connections 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 -- 172.21.15.86:0/2336809304 wait complete. 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 -- start start 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.018+0000 7fda1aec2b80 1 Processor -- start 2024-03-22T19:11:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfdadbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfdae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd56160 0x55a9dfdb5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfdb55b0 con 0x55a9dfd56160 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfdb56f0 con 0x55a9dfd4ed90 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9dfdb5880 con 0x55a9dfd4fc30 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda1aec2b80 10 client.0 _ll_register_callbacks cb 0x7fd9f40994b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd56160 0x55a9dfdb5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fd9ffe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfdae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fd9ffe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfdae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54700/0 (socket says 172.21.15.86:54700) 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fd9ffe00700 1 -- 172.21.15.86:0/3796824317 learned_addr learned my addr 172.21.15.86:0/3796824317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.019+0000 7fda04a00700 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfdadbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda04a00700 1 -- 172.21.15.86:0/3796824317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 msgr2=0x55a9dfdae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda04a00700 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfdae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda04a00700 1 -- 172.21.15.86:0/3796824317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd56160 msgr2=0x55a9dfdb5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda04a00700 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd56160 0x55a9dfdb5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda04a00700 1 -- 172.21.15.86:0/3796824317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9dfac5b60 con 0x55a9dfd4ed90 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda04a00700 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfdadbe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4099620 tx=0x7fd9f405fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70912556d04cc56 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fd9e6000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fd9e6000700 1 -- 172.21.15.86:0/3796824317 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40a1df0 con 0x55a9dfd4ed90 2024-03-22T19:11:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fd9e6000700 1 -- 172.21.15.86:0/3796824317 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fd9f40b22f0 con 0x55a9dfd4ed90 2024-03-22T19:11:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f409d7a0 con 0x55a9dfd4ed90 2024-03-22T19:11:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fd9e6000700 1 -- 172.21.15.86:0/3796824317 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f4098a40 con 0x55a9dfd4ed90 2024-03-22T19:11:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda1aec2b80 10 client.25562 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.020+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd9ec009db0 con 0x55a9dfd4ed90 2024-03-22T19:11:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fd9e6000700 1 -- 172.21.15.86:0/3796824317 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f409d7a0 con 0x55a9dfd4ed90 2024-03-22T19:11:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fd9e6000700 1 -- 172.21.15.86:0/3796824317 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd9f4095760 con 0x55a9dfd4ed90 2024-03-22T19:11:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fda1aec2b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fda1aec2b80 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:57.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fda1aec2b80 20 client.25562 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:57.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfdb6e00 0x55a9dfdb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fda1aec2b80 4 client.25562 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:11:57.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9dfae6cf0 con 0x55a9dfdb6e00 2024-03-22T19:11:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.021+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfdb6e00 0x55a9dfdb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fd9ffe00700 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfdb6e00 0x55a9dfdb9220 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9795150c9adfe224 server_cookie=2cea81467bf12340 in_seq=0 out_seq=0 2024-03-22T19:11:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fd9e6000700 10 client.25562 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fd9e6000700 1 -- 172.21.15.86:0/3796824317 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a9dfae6cf0 con 0x55a9dfdb6e00 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fd9e6000700 10 client.25562 handle_command_reply: tid=0 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 client.25562 shutdown 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 msgr2=0x55a9dfdadbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfdadbe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4099620 tx=0x7fd9f405fce0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 shutdown_connections 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55a9dfdb6e00 0x55a9dfdb9220 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9dfd56160 0x55a9dfdb5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9dfd4ed90 0x55a9dfdadbe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fd9e6000700 0 client.25562 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 --2- 172.21.15.86:0/3796824317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9dfd4fc30 0x55a9dfdae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.022+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 >> 172.21.15.86:0/3796824317 conn(0x55a9dfcbb220 msgr2=0x55a9dfd40aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 shutdown_connections 2024-03-22T19:11:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda1aec2b80 1 -- 172.21.15.86:0/3796824317 wait complete. 2024-03-22T19:11:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda1aec2b80 20 client.25562 trim_cache size 0 max 16384 2024-03-22T19:11:57.033 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:11:57.033 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:11:57.033 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:11:57.033 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:11:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd9e80a0a90 msgr2=0x7fd9e80a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda07400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd9e80a0a90 0x7fd9e80a2f50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009fd0 tx=0x7fd9f8007680 comp rx=0 tx=0).stop 2024-03-22T19:11:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 msgr2=0x7fda00198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.023+0000 7fda07400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 0x7fda00198fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400e9a0 tx=0x7fd9f400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 shutdown_connections 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd9e80a0a90 0x7fd9e80a2f50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda001008c0 0x7fda001a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda000ff5a0 0x7fda00198fa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 --2- 172.21.15.86:0/650094421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda000fff00 0x7fda001994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 >> 172.21.15.86:0/650094421 conn(0x7fda000fb1a0 msgr2=0x7fda000fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 shutdown_connections 2024-03-22T19:11:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.024+0000 7fda07400700 1 -- 172.21.15.86:0/650094421 wait complete. 2024-03-22T19:11:57.050 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:11:57.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 -- 172.21.15.86:0/1695288036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 msgr2=0x7fba60106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 --2- 172.21.15.86:0/1695288036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 0x7fba60106450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009990 tx=0x7fba4c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 -- 172.21.15.86:0/1695288036 shutdown_connections 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 --2- 172.21.15.86:0/1695288036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 0x7fba60108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 --2- 172.21.15.86:0/1695288036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 --2- 172.21.15.86:0/1695288036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 0x7fba60106450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 -- 172.21.15.86:0/1695288036 >> 172.21.15.86:0/1695288036 conn(0x7fba600fb060 msgr2=0x7fba600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 -- 172.21.15.86:0/1695288036 shutdown_connections 2024-03-22T19:11:57.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.252+0000 7fba64e00700 1 -- 172.21.15.86:0/1695288036 wait complete. 2024-03-22T19:11:57.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.253+0000 7fba64e00700 1 Processor -- start 2024-03-22T19:11:57.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 -- start start 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 0x7fba601992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 0x7fba601a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a0870 con 0x7fba60101730 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a09e0 con 0x7fba60104060 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba64e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a0ba0 con 0x7fba60106990 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba5f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba5fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 0x7fba601a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba5f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54902/0 (socket says 172.21.15.86:54902) 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba5f400700 1 -- 172.21.15.86:0/2408474117 learned_addr learned my addr 172.21.15.86:0/2408474117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.254+0000 7fba5fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 0x7fba601a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54726/0 (socket says 172.21.15.86:54726) 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5ea00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 0x7fba601992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5f400700 1 -- 172.21.15.86:0/2408474117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 msgr2=0x7fba601a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5f400700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 0x7fba601a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5f400700 1 -- 172.21.15.86:0/2408474117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 msgr2=0x7fba601992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5f400700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 0x7fba601992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5f400700 1 -- 172.21.15.86:0/2408474117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0095f0 con 0x7fba60101730 2024-03-22T19:11:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.255+0000 7fba5f400700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60198d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fba5000d950 tx=0x7fba5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.256+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000f890 con 0x7fba60101730 2024-03-22T19:11:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.256+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba50010460 con 0x7fba60101730 2024-03-22T19:11:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.256+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000e610 con 0x7fba60101730 2024-03-22T19:11:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.256+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba601a0e80 con 0x7fba60101730 2024-03-22T19:11:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.256+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601a1250 con 0x7fba60101730 2024-03-22T19:11:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.257+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba60108510 con 0x7fba60101730 2024-03-22T19:11:57.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.259+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fba500108d0 con 0x7fba60101730 2024-03-22T19:11:57.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.260+0000 7fba57e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fba440a0b10 0x7fba440a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.260+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fba500cbd60 con 0x7fba60101730 2024-03-22T19:11:57.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.261+0000 7fba5ea00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fba440a0b10 0x7fba440a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.261+0000 7fba5ea00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fba440a0b10 0x7fba440a2fd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00b5c0 tx=0x7fba4c003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.265+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fba50095720 con 0x7fba60101730 2024-03-22T19:11:57.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:57 smithi131 ceph-mon[78559]: pgmap v315: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:57.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:57 smithi086 ceph-mon[38032]: pgmap v315: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.532+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba60067970 con 0x7fba60101730 2024-03-22T19:11:57.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.533+0000 7fba57e00700 1 -- 172.21.15.86:0/2408474117 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10924 (secure 0 0 0) 0x7fba50098d40 con 0x7fba60101730 2024-03-22T19:11:57.535 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:11:57.535 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:10:44.254849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:11:57.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.536+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fba440a0b10 msgr2=0x7fba440a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.536+0000 7fba64e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fba440a0b10 0x7fba440a2fd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00b5c0 tx=0x7fba4c003790 comp rx=0 tx=0).stop 2024-03-22T19:11:57.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 msgr2=0x7fba60198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60198d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fba5000d950 tx=0x7fba5000dc60 comp rx=0 tx=0).stop 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 shutdown_connections 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fba440a0b10 0x7fba440a2fd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba60106990 0x7fba601a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60101730 0x7fba60198d80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 --2- 172.21.15.86:0/2408474117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba60104060 0x7fba601992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 >> 172.21.15.86:0/2408474117 conn(0x7fba600fb060 msgr2=0x7fba60100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 shutdown_connections 2024-03-22T19:11:57.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.537+0000 7fba64e00700 1 -- 172.21.15.86:0/2408474117 wait complete. 2024-03-22T19:11:57.540 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-22T19:11:57.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:11:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:57 smithi167 ceph-mon[47728]: pgmap v315: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:11:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 -- 172.21.15.86:0/2302548891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0101d70 msgr2=0x7ffba01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 --2- 172.21.15.86:0/2302548891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0101d70 0x7ffba01021d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0099c0 tx=0x7ffb8c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 -- 172.21.15.86:0/2302548891 shutdown_connections 2024-03-22T19:11:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 --2- 172.21.15.86:0/2302548891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba0102710 0x7ffba0108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 --2- 172.21.15.86:0/2302548891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffba0100b80 0x7ffba0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 --2- 172.21.15.86:0/2302548891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0101d70 0x7ffba01021d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.795+0000 7ffba4e00700 1 -- 172.21.15.86:0/2302548891 >> 172.21.15.86:0/2302548891 conn(0x7ffba00fc0f0 msgr2=0x7ffba00fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.796+0000 7ffba4e00700 1 -- 172.21.15.86:0/2302548891 shutdown_connections 2024-03-22T19:11:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.796+0000 7ffba4e00700 1 -- 172.21.15.86:0/2302548891 wait complete. 2024-03-22T19:11:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.797+0000 7ffba4e00700 1 Processor -- start 2024-03-22T19:11:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.797+0000 7ffba4e00700 1 -- start start 2024-03-22T19:11:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.797+0000 7ffba4e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 0x7ffba0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.797+0000 7ffba4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffba0101d70 0x7ffba01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffba4e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba0102710 0x7ffba01a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffba4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba01a0890 con 0x7ffba0101d70 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffba4e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba01a0a00 con 0x7ffba0100b80 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffba4e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba01a0bc0 con 0x7ffba0102710 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 0x7ffba0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 0x7ffba0198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44940/0 (socket says 172.21.15.86:44940) 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 -- 172.21.15.86:0/831195316 learned_addr learned my addr 172.21.15.86:0/831195316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffba0101d70 0x7ffba01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba0102710 0x7ffba01a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 -- 172.21.15.86:0/831195316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba0102710 msgr2=0x7ffba01a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba0102710 0x7ffba01a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 -- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffba0101d70 msgr2=0x7ffba01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffba0101d70 0x7ffba01992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.798+0000 7ffb9f400700 1 -- 172.21.15.86:0/831195316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0095f0 con 0x7ffba0100b80 2024-03-22T19:11:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.799+0000 7ffb9f400700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 0x7ffba0198da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000e970 tx=0x7ffb9000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.799+0000 7ffb97e00700 1 -- 172.21.15.86:0/831195316 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90009790 con 0x7ffba0100b80 2024-03-22T19:11:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.799+0000 7ffb97e00700 1 -- 172.21.15.86:0/831195316 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb9000eec0 con 0x7ffba0100b80 2024-03-22T19:11:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.799+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba01a0ea0 con 0x7ffba0100b80 2024-03-22T19:11:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.799+0000 7ffb97e00700 1 -- 172.21.15.86:0/831195316 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90003cf0 con 0x7ffba0100b80 2024-03-22T19:11:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.799+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba01a12c0 con 0x7ffba0100b80 2024-03-22T19:11:57.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.803+0000 7ffbb9f9db80 1 Processor -- start 2024-03-22T19:11:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 -- start start 2024-03-22T19:11:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffb97e00700 1 -- 172.21.15.86:0/831195316 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ffb9000ccd0 con 0x7ffba0100b80 2024-03-22T19:11:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df0f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df043990 con 0x5558df0f35f0 2024-03-22T19:11:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df041350 con 0x5558df0f51f0 2024-03-22T19:11:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df03cc40 con 0x5558df0f4830 2024-03-22T19:11:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df0f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.804+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df0f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54768/0 (socket says 172.21.15.86:54768) 2024-03-22T19:11:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9f400700 1 -- 172.21.15.86:0/4055875921 learned_addr learned my addr 172.21.15.86:0/4055875921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9f400700 1 -- 172.21.15.86:0/4055875921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 msgr2=0x5558df0fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9f400700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9f400700 1 -- 172.21.15.86:0/4055875921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 msgr2=0x5558df0f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9f400700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9f400700 1 -- 172.21.15.86:0/4055875921 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558deebe3b0 con 0x5558df0f4830 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.805+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb97e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffb840a0a90 0x7ffb840a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb9f400700 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df0f4cb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cffa0 tx=0x7ffb900d40e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd85bfb3b9c65c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb97e00700 1 -- 172.21.15.86:0/831195316 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ffb90014070 con 0x7ffba0100b80 2024-03-22T19:11:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb94c00700 1 -- 172.21.15.86:0/4055875921 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90095c50 con 0x5558df0f4830 2024-03-22T19:11:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb94c00700 1 -- 172.21.15.86:0/4055875921 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb900996e0 con 0x5558df0f4830 2024-03-22T19:11:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb94c00700 1 -- 172.21.15.86:0/4055875921 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9009a010 con 0x5558df0f4830 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.806+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffb840a0a90 0x7ffb840a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.807+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/4055875921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 msgr2=0x5558df0f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.807+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df0f4cb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cffa0 tx=0x7ffb900d40e0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.807+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffb840a0a90 0x7ffb840a2f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5558df0f3390 tx=0x7ffb9800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.807+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/4055875921 shutdown_connections 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.807+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/4055875921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df0f4cb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/4055875921 >> 172.21.15.86:0/4055875921 conn(0x5558df060220 msgr2=0x5558df0e5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/4055875921 shutdown_connections 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/4055875921 wait complete. 2024-03-22T19:11:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 -- start start 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.808+0000 7ffbb9f9db80 1 Processor -- start 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f4830 0x5558df0fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f51f0 0x5558df104a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558defd8d80 con 0x5558df0f35f0 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558defd8a20 con 0x5558df0f4830 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deec06a0 con 0x5558df0f51f0 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffbb9f9db80 10 client.0 _ll_register_callbacks cb 0x5558dee443a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.809+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54940/0 (socket says 172.21.15.86:54940) 2024-03-22T19:11:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 -- 172.21.15.86:0/1629131751 learned_addr learned my addr 172.21.15.86:0/1629131751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f4830 0x5558df0fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 -- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f51f0 msgr2=0x5558df104a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f51f0 0x5558df104a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f51f0 0x5558df104a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 -- 172.21.15.86:0/1629131751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f4830 msgr2=0x5558df0fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f4830 0x5558df0fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 -- 172.21.15.86:0/1629131751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb98009050 con 0x5558df0f35f0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f51f0 0x5558df104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb9f400700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0fd4f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cff70 tx=0x7ffb900cfa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9286c4e951bac9d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb89600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb89600700 1 -- 172.21.15.86:0/1629131751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90095c50 con 0x5558df0f35f0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.810+0000 7ffb89600700 1 -- 172.21.15.86:0/1629131751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ffb9009a010 con 0x5558df0f35f0 2024-03-22T19:11:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558df0ddc30 con 0x5558df0f35f0 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffb89600700 1 -- 172.21.15.86:0/1629131751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb90099910 con 0x5558df0f35f0 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 10 client.15561 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ffba01a12c0 con 0x5558df0f35f0 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffb89600700 1 -- 172.21.15.86:0/1629131751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558df0ddc30 con 0x5558df0f35f0 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffb89600700 1 -- 172.21.15.86:0/1629131751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffb90099b70 con 0x5558df0f35f0 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 10 client.15561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 20 client.15561 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df108e40 0x5558df10b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 4 client.15561 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.811+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558df035700 con 0x5558df108e40 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.812+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df108e40 0x5558df10b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.812+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df108e40 0x5558df10b260 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c60483e78cd0f306 server_cookie=cc22b94346da383f in_seq=0 out_seq=0 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.812+0000 7ffb89600700 10 client.15561 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.813+0000 7ffb89600700 1 -- 172.21.15.86:0/1629131751 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5558df035700 con 0x5558df108e40 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.813+0000 7ffb89600700 10 client.15561 handle_command_reply: tid=0 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.813+0000 7ffbb9f9db80 1 client.15561 shutdown 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 msgr2=0x5558df0fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0fd4f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cff70 tx=0x7ffb900cfa50 comp rx=0 tx=0).stop 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 shutdown_connections 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df108e40 0x5558df10b260 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f51f0 0x5558df104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f35f0 0x5558df0fd4f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/1629131751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f4830 0x5558df0fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 >> 172.21.15.86:0/1629131751 conn(0x5558df060220 msgr2=0x5558df0e5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffb89600700 0 client.15561 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 shutdown_connections 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.814+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/1629131751 wait complete. 2024-03-22T19:11:57.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.815+0000 7ffbb9f9db80 20 client.15561 trim_cache size 0 max 16384 2024-03-22T19:11:57.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 Processor -- start 2024-03-22T19:11:57.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 -- start start 2024-03-22T19:11:57.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f35f0 0x5558df1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f4830 0x5558df1050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df0f3c40 con 0x5558df0f4830 2024-03-22T19:11:57.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df0e03c0 con 0x5558df0f51f0 2024-03-22T19:11:57.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df105ba0 con 0x5558df0f35f0 2024-03-22T19:11:57.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.828+0000 7ffb9ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f4830 0x5558df1050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f4830 0x5558df1050d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54948/0 (socket says 172.21.15.86:54948) 2024-03-22T19:11:57.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9ea00700 1 -- 172.21.15.86:0/217546550 learned_addr learned my addr 172.21.15.86:0/217546550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f35f0 0x5558df1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 -- 172.21.15.86:0/217546550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 msgr2=0x5558df0fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 -- 172.21.15.86:0/217546550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f4830 msgr2=0x5558df1050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f4830 0x5558df1050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 -- 172.21.15.86:0/217546550 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558dee6ab60 con 0x5558df0f35f0 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb9f400700 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f35f0 0x5558df1029d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffb900998a0 tx=0x7ffb900c9e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481d74ffeb642292 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.829+0000 7ffb95600700 1 -- 172.21.15.86:0/217546550 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90099270 con 0x5558df0f35f0 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffb95600700 1 -- 172.21.15.86:0/217546550 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb90095c50 con 0x5558df0f35f0 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffb95600700 1 -- 172.21.15.86:0/217546550 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90099910 con 0x5558df0f35f0 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/217546550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f35f0 msgr2=0x5558df1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f35f0 0x5558df1029d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffb900998a0 tx=0x7ffb900c9e00 comp rx=0 tx=0).stop 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/217546550 shutdown_connections 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f51f0 0x5558df0fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f35f0 0x5558df1029d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/217546550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f4830 0x5558df1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/217546550 >> 172.21.15.86:0/217546550 conn(0x5558df060220 msgr2=0x5558df0fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.830+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/217546550 shutdown_connections 2024-03-22T19:11:57.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.831+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/217546550 wait complete. 2024-03-22T19:11:57.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.831+0000 7ffbb9f9db80 1 -- start start 2024-03-22T19:11:57.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.831+0000 7ffbb9f9db80 1 Processor -- start 2024-03-22T19:11:57.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 0x5558df102140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df102680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f51f0 0x5558df159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df1595b0 con 0x5558df0f51f0 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df159720 con 0x5558df0f35f0 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558df1598e0 con 0x5558df0f4830 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffbb9f9db80 10 client.0 _ll_register_callbacks cb 0x7ffb98009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 0x5558df102140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffb9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 0x5558df102140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44970/0 (socket says 172.21.15.86:44970) 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffb9f400700 1 -- 172.21.15.86:0/2538967164 learned_addr learned my addr 172.21.15.86:0/2538967164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.832+0000 7ffb9fe00700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f51f0 0x5558df159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb9f400700 1 -- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 msgr2=0x5558df102680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb9f400700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df102680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb9f400700 1 -- 172.21.15.86:0/2538967164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f51f0 msgr2=0x5558df159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb9f400700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f51f0 0x5558df159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb9f400700 1 -- 172.21.15.86:0/2538967164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba01a12c0 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb9f400700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 0x5558df102140 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffb90095e10 tx=0x7ffb9000c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327d6bcb466b026a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb89600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb89600700 1 -- 172.21.15.86:0/2538967164 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90099270 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb89600700 1 -- 172.21.15.86:0/2538967164 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ffb90097d70 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb900996e0 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffb89600700 1 -- 172.21.15.86:0/2538967164 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb90095510 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffbb9f9db80 10 client.25586 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.833+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5558df002ea0 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffb89600700 1 -- 172.21.15.86:0/2538967164 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb900a5020 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffb89600700 1 -- 172.21.15.86:0/2538967164 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffb90098e00 con 0x5558df0f35f0 2024-03-22T19:11:57.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffbb9f9db80 10 client.25586 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:57.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffbb9f9db80 10 client.25586 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:11:57.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffbb9f9db80 20 client.25586 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:57.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df15ac90 0x5558df15d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:57.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffbb9f9db80 4 client.25586 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:57.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5558dee8bcf0 con 0x5558df15ac90 2024-03-22T19:11:57.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.834+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df15ac90 0x5558df15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:57.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.835+0000 7ffb89600700 10 client.25586 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:57.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.835+0000 7ffb9ea00700 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df15ac90 0x5558df15d080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a67b0b43167db9a server_cookie=2aa2664b6d4ae35b in_seq=0 out_seq=0 2024-03-22T19:11:57.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.835+0000 7ffb89600700 1 -- 172.21.15.86:0/2538967164 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5558dee8bcf0 con 0x5558df15ac90 2024-03-22T19:11:57.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.835+0000 7ffb89600700 10 client.25586 handle_command_reply: tid=0 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.835+0000 7ffbb9f9db80 1 client.25586 shutdown 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 msgr2=0x5558df102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 0x5558df102140 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffb90095e10 tx=0x7ffb9000c6e0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 shutdown_connections 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558df0f51f0 0x5558df159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558df0f35f0 0x5558df102140 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5558df15ac90 0x5558df15d080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 --2- 172.21.15.86:0/2538967164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558df0f4830 0x5558df102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffb89600700 0 client.25586 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 >> 172.21.15.86:0/2538967164 conn(0x5558df060220 msgr2=0x5558df0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 shutdown_connections 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 1 -- 172.21.15.86:0/2538967164 wait complete. 2024-03-22T19:11:57.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.836+0000 7ffbb9f9db80 20 client.25586 trim_cache size 0 max 16384 2024-03-22T19:11:57.841 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:57.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffb840a0a90 msgr2=0x7ffb840a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffb840a0a90 0x7ffb840a2f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5558df0f3390 tx=0x7ffb9800b040 comp rx=0 tx=0).stop 2024-03-22T19:11:57.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 msgr2=0x7ffba0198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:57.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 0x7ffba0198da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000e970 tx=0x7ffb9000ec80 comp rx=0 tx=0).stop 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 shutdown_connections 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ffb840a0a90 0x7ffb840a2f50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba0102710 0x7ffba01a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba0100b80 0x7ffba0198da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 --2- 172.21.15.86:0/831195316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffba0101d70 0x7ffba01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.837+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 >> 172.21.15.86:0/831195316 conn(0x7ffba00fc0f0 msgr2=0x7ffba0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.838+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 shutdown_connections 2024-03-22T19:11:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:57.838+0000 7ffba4e00700 1 -- 172.21.15.86:0/831195316 wait complete. 2024-03-22T19:11:57.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:11:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 -- 172.21.15.86:0/332845008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141017b0 msgr2=0x7f8b14103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 --2- 172.21.15.86:0/332845008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141017b0 0x7f8b14103ba0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009990 tx=0x7f8b04009ca0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 -- 172.21.15.86:0/332845008 shutdown_connections 2024-03-22T19:11:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 --2- 172.21.15.86:0/332845008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b14106a10 0x7f8b14108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 --2- 172.21.15.86:0/332845008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141017b0 0x7f8b14103ba0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 --2- 172.21.15.86:0/332845008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b141040e0 0x7f8b141064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 -- 172.21.15.86:0/332845008 >> 172.21.15.86:0/332845008 conn(0x7f8b140fb1a0 msgr2=0x7f8b140fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 -- 172.21.15.86:0/332845008 shutdown_connections 2024-03-22T19:11:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.062+0000 7f8b19200700 1 -- 172.21.15.86:0/332845008 wait complete. 2024-03-22T19:11:58.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.063+0000 7f8b19200700 1 Processor -- start 2024-03-22T19:11:58.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.063+0000 7f8b19200700 1 -- start start 2024-03-22T19:11:58.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b19200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 0x7f8b14196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b19200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141040e0 0x7f8b141971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b19200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b14106a10 0x7f8b1419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b19200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1419e790 con 0x7f8b141017b0 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b19200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1419e900 con 0x7f8b141040e0 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b19200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140ff5a0 con 0x7f8b14106a10 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 0x7f8b14196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 0x7f8b14196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54986/0 (socket says 172.21.15.86:54986) 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b11e00700 1 -- 172.21.15.86:0/1477424145 learned_addr learned my addr 172.21.15.86:0/1477424145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b12800700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b14106a10 0x7f8b1419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.064+0000 7f8b11400700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141040e0 0x7f8b141971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b11e00700 1 -- 172.21.15.86:0/1477424145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b14106a10 msgr2=0x7f8b1419e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b11e00700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b14106a10 0x7f8b1419e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b11e00700 1 -- 172.21.15.86:0/1477424145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141040e0 msgr2=0x7f8b141971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b11e00700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141040e0 0x7f8b141971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b11e00700 1 -- 172.21.15.86:0/1477424145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040095f0 con 0x7f8b141017b0 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b12800700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b14106a10 0x7f8b1419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b11e00700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 0x7f8b14196ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005910 tx=0x7f8b04009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b0aa00700 1 -- 172.21.15.86:0/1477424145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0401d070 con 0x7f8b141017b0 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b0aa00700 1 -- 172.21.15.86:0/1477424145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b04021410 con 0x7f8b141017b0 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.065+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b140ff820 con 0x7f8b141017b0 2024-03-22T19:11:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.066+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b140ffcd0 con 0x7f8b141017b0 2024-03-22T19:11:58.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.066+0000 7f8b0aa00700 1 -- 172.21.15.86:0/1477424145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04005720 con 0x7f8b141017b0 2024-03-22T19:11:58.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.070+0000 7f8b0aa00700 1 -- 172.21.15.86:0/1477424145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8b04021580 con 0x7f8b141017b0 2024-03-22T19:11:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.072+0000 7f8b2cd39b80 1 Processor -- start 2024-03-22T19:11:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.072+0000 7f8b0aa00700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8af80a0a90 0x7f8af80a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.072+0000 7f8b0aa00700 1 -- 172.21.15.86:0/1477424145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b040cd2e0 con 0x7f8b141017b0 2024-03-22T19:11:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 -- start start 2024-03-22T19:11:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b12800700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8af80a0a90 0x7f8af80a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be1d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be1de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881bd75990 con 0x55881be1e420 2024-03-22T19:11:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881bd73350 con 0x55881be1cc20 2024-03-22T19:11:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881bd6ec40 con 0x55881be1d580 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b11400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be1de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b11400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be1de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54834/0 (socket says 172.21.15.86:54834) 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.073+0000 7f8b11400700 1 -- 172.21.15.86:0/5240717 learned_addr learned my addr 172.21.15.86:0/5240717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11e00700 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be1d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b12800700 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11400700 1 -- 172.21.15.86:0/5240717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 msgr2=0x55881be1d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11400700 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be1d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11400700 1 -- 172.21.15.86:0/5240717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 msgr2=0x55881be1e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11400700 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11400700 1 -- 172.21.15.86:0/5240717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881bbf03b0 con 0x55881be1d580 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b11400700 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be1de40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009d80 tx=0x7f8b00011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91e3e205c8b518f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8b12800700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8af80a0a90 0x7f8af80a2f50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c004230 tx=0x7f8b0c0045f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.074+0000 7f8aff400700 1 -- 172.21.15.86:0/5240717 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00019630 con 0x55881be1d580 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.075+0000 7f8aff400700 1 -- 172.21.15.86:0/5240717 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b00019c50 con 0x55881be1d580 2024-03-22T19:11:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.075+0000 7f8aff400700 1 -- 172.21.15.86:0/5240717 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000ebb0 con 0x55881be1d580 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/5240717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 msgr2=0x55881be1de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be1de40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009d80 tx=0x7f8b00011720 comp rx=0 tx=0).stop 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/5240717 shutdown_connections 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be1d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/5240717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be1de40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/5240717 >> 172.21.15.86:0/5240717 conn(0x55881bd92220 msgr2=0x55881be17bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/5240717 shutdown_connections 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/5240717 wait complete. 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.076+0000 7f8b2cd39b80 1 -- start start 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 Processor -- start 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1d580 0x55881be27040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1e420 0x55881be2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881bd0ad80 con 0x55881be1cc20 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881bd0aa20 con 0x55881be1d580 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881bbf26a0 con 0x55881be1e420 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.077+0000 7f8b2cd39b80 10 client.0 _ll_register_callbacks cb 0x55881bb763a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b12800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1e420 0x55881be2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b12800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1e420 0x55881be2e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54838/0 (socket says 172.21.15.86:54838) 2024-03-22T19:11:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b12800700 1 -- 172.21.15.86:0/1936413206 learned_addr learned my addr 172.21.15.86:0/1936413206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 -- 172.21.15.86:0/1936413206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1e420 msgr2=0x55881be2e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1e420 0x55881be2e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 -- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1d580 msgr2=0x55881be27040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1d580 0x55881be27040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 -- 172.21.15.86:0/1936413206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b000099e0 con 0x55881be1cc20 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.078+0000 7f8b11e00700 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be26b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b040caec0 tx=0x7f8b04021b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1c0fa0461e1298 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8af3e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8af3e00700 1 -- 172.21.15.86:0/1936413206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0409a180 con 0x55881be1cc20 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8af3e00700 1 -- 172.21.15.86:0/1936413206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b0409e8c0 con 0x55881be1cc20 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55881be0fad0 con 0x55881be1cc20 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8af3e00700 1 -- 172.21.15.86:0/1936413206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b040a2e80 con 0x55881be1cc20 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8b2cd39b80 10 client.15579 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.079+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55881bb9cb60 con 0x55881be1cc20 2024-03-22T19:11:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.080+0000 7f8af3e00700 1 -- 172.21.15.86:0/1936413206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55881be0fad0 con 0x55881be1cc20 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.080+0000 7f8af3e00700 1 -- 172.21.15.86:0/1936413206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b04096880 con 0x55881be1cc20 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b2cd39b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b2cd39b80 10 client.15579 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b2cd39b80 20 client.15579 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be324a0 0x55881be348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b2cd39b80 4 client.15579 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55881bd67700 con 0x55881be324a0 2024-03-22T19:11:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.081+0000 7f8b11400700 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be324a0 0x55881be348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.082+0000 7f8af3e00700 10 client.15579 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.082+0000 7f8b11400700 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be324a0 0x55881be348c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88440f24b76e655e server_cookie=a42b773a3295da5a in_seq=0 out_seq=0 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8af3e00700 1 -- 172.21.15.86:0/1936413206 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55881bd67700 con 0x55881be324a0 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8af3e00700 10 client.15579 handle_command_reply: tid=0 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 client.15579 shutdown 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 msgr2=0x55881be26b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be26b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b040caec0 tx=0x7f8b04021b40 comp rx=0 tx=0).stop 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 shutdown_connections 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be324a0 0x55881be348c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1e420 0x55881be2e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be26b00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/1936413206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1d580 0x55881be27040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 >> 172.21.15.86:0/1936413206 conn(0x55881bd92220 msgr2=0x55881be171f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.083+0000 7f8af3e00700 0 client.15579 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.084+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 shutdown_connections 2024-03-22T19:11:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.084+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/1936413206 wait complete. 2024-03-22T19:11:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.084+0000 7f8b2cd39b80 20 client.15579 trim_cache size 0 max 16384 2024-03-22T19:11:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.097+0000 7f8b2cd39b80 1 Processor -- start 2024-03-22T19:11:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.097+0000 7f8b2cd39b80 1 -- start start 2024-03-22T19:11:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881be1d990 con 0x55881be1e420 2024-03-22T19:11:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881be0f640 con 0x55881be1cc20 2024-03-22T19:11:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881be1d440 con 0x55881be1d580 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be391e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:45002/0 (socket says 172.21.15.86:45002) 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b12800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.098+0000 7f8b11e00700 1 -- 172.21.15.86:0/394121816 learned_addr learned my addr 172.21.15.86:0/394121816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11400700 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11e00700 1 -- 172.21.15.86:0/394121816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 msgr2=0x55881be2dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11e00700 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11e00700 1 -- 172.21.15.86:0/394121816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 msgr2=0x55881be1e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11e00700 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11e00700 1 -- 172.21.15.86:0/394121816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881bd34ea0 con 0x55881be1cc20 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b12800700 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11400700 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8b11e00700 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be391e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b04096f90 tx=0x7f8b04099040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec043afc63808aa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8affe00700 1 -- 172.21.15.86:0/394121816 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040a2e80 con 0x55881be1cc20 2024-03-22T19:11:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8affe00700 1 -- 172.21.15.86:0/394121816 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b04096880 con 0x55881be1cc20 2024-03-22T19:11:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.099+0000 7f8affe00700 1 -- 172.21.15.86:0/394121816 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040b3120 con 0x55881be1cc20 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/394121816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 msgr2=0x55881be391e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be391e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b04096f90 tx=0x7f8b04099040 comp rx=0 tx=0).stop 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/394121816 shutdown_connections 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1e420 0x55881be1e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1cc20 0x55881be391e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/394121816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.100+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/394121816 >> 172.21.15.86:0/394121816 conn(0x55881bd92220 msgr2=0x55881be26010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.101+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/394121816 shutdown_connections 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.101+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/394121816 wait complete. 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.101+0000 7f8b2cd39b80 1 -- start start 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.101+0000 7f8b2cd39b80 1 Processor -- start 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be29e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1e420 0x55881be2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881be2d880 con 0x55881be1cc20 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881be2d9f0 con 0x55881be1e420 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881be2dbb0 con 0x55881be1d580 2024-03-22T19:11:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b2cd39b80 10 client.0 _ll_register_callbacks cb 0x7f8b0409a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be29e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be29e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55026/0 (socket says 172.21.15.86:55026) 2024-03-22T19:11:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b11e00700 1 -- 172.21.15.86:0/826387384 learned_addr learned my addr 172.21.15.86:0/826387384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b11400700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.102+0000 7f8b12800700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1e420 0x55881be2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11e00700 1 -- 172.21.15.86:0/826387384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 msgr2=0x55881be2a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11e00700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11e00700 1 -- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1e420 msgr2=0x55881be2d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11e00700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1e420 0x55881be2d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11e00700 1 -- 172.21.15.86:0/826387384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55881bb9cb60 con 0x55881be1cc20 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11400700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8b11e00700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be29e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b680 tx=0x7f8b04021c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb12571cf60984d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.103+0000 7f8af3e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8af3e00700 1 -- 172.21.15.86:0/826387384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040969f0 con 0x55881be1cc20 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8af3e00700 1 -- 172.21.15.86:0/826387384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b040a2e80 con 0x55881be1cc20 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b00019c50 con 0x55881be1cc20 2024-03-22T19:11:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8af3e00700 1 -- 172.21.15.86:0/826387384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b04060de0 con 0x55881be1cc20 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8b2cd39b80 10 client.15585 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55881bbf4cd0 con 0x55881be1cc20 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8af3e00700 1 -- 172.21.15.86:0/826387384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b00019c50 con 0x55881be1cc20 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.104+0000 7f8af3e00700 1 -- 172.21.15.86:0/826387384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b04099b10 con 0x55881be1cc20 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b2cd39b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b2cd39b80 10 client.15585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b2cd39b80 20 client.15585 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be8cfd0 0x55881be8f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b2cd39b80 4 client.15585 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55881bbbdcf0 con 0x55881be8cfd0 2024-03-22T19:11:58.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.105+0000 7f8b11400700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be8cfd0 0x55881be8f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8af3e00700 10 client.15585 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:58.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8b11400700 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be8cfd0 0x55881be8f3c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2da16a46ca3cfc3 server_cookie=1653e8d1e4651b42 in_seq=0 out_seq=0 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8af3e00700 1 -- 172.21.15.86:0/826387384 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55881bbbdcf0 con 0x55881be8cfd0 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8af3e00700 10 client.15585 handle_command_reply: tid=0 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8b2cd39b80 1 client.15585 shutdown 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 msgr2=0x55881be29e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.106+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be29e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b680 tx=0x7f8b04021c00 comp rx=0 tx=0).stop 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 shutdown_connections 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55881be1e420 0x55881be2d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55881be1cc20 0x55881be29e80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55881be8cfd0 0x55881be8f3c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 --2- 172.21.15.86:0/826387384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881be1d580 0x55881be2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8af3e00700 0 client.15585 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 >> 172.21.15.86:0/826387384 conn(0x55881bd92220 msgr2=0x55881be24570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 shutdown_connections 2024-03-22T19:11:58.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 1 -- 172.21.15.86:0/826387384 wait complete. 2024-03-22T19:11:58.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.107+0000 7f8b2cd39b80 20 client.15585 trim_cache size 0 max 16384 2024-03-22T19:11:58.111 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8af80a0a90 msgr2=0x7f8af80a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8af80a0a90 0x7f8af80a2f50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c004230 tx=0x7f8b0c0045f0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 msgr2=0x7f8b14196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 0x7f8b14196ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005910 tx=0x7f8b04009580 comp rx=0 tx=0).stop 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 shutdown_connections 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8af80a0a90 0x7f8af80a2f50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b14106a10 0x7f8b1419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b141017b0 0x7f8b14196ca0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 --2- 172.21.15.86:0/1477424145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b141040e0 0x7f8b141971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.108+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 >> 172.21.15.86:0/1477424145 conn(0x7f8b140fb1a0 msgr2=0x7f8b14103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.109+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 shutdown_connections 2024-03-22T19:11:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.109+0000 7f8b19200700 1 -- 172.21.15.86:0/1477424145 wait complete. 2024-03-22T19:11:58.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:11:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 -- 172.21.15.86:0/3336790268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ffdb0 msgr2=0x7fd6b0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/3336790268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0100230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0099c0 tx=0x7fd69c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 -- 172.21.15.86:0/3336790268 shutdown_connections 2024-03-22T19:11:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/3336790268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6b0100770 0x7fd6b0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/3336790268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ff450 0x7fd6b00ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/3336790268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0100230 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.355+0000 7fd6b5a00700 1 -- 172.21.15.86:0/3336790268 >> 172.21.15.86:0/3336790268 conn(0x7fd6b00fafb0 msgr2=0x7fd6b00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.356+0000 7fd6b5a00700 1 -- 172.21.15.86:0/3336790268 shutdown_connections 2024-03-22T19:11:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.356+0000 7fd6b5a00700 1 -- 172.21.15.86:0/3336790268 wait complete. 2024-03-22T19:11:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 Processor -- start 2024-03-22T19:11:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 -- start start 2024-03-22T19:11:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ff450 0x7fd6b0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6b0100770 0x7fd6b01a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01a0800 con 0x7fd6b00ff450 2024-03-22T19:11:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01a0970 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.357+0000 7fd6b5a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01a0b30 con 0x7fd6b0100770 2024-03-22T19:11:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ff450 0x7fd6b0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:45022/0 (socket says 172.21.15.86:45022) 2024-03-22T19:11:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 -- 172.21.15.86:0/2085027870 learned_addr learned my addr 172.21.15.86:0/2085027870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6b5000700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6b0100770 0x7fd6b01a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 -- 172.21.15.86:0/2085027870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6b0100770 msgr2=0x7fd6b01a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6b0100770 0x7fd6b01a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 -- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ff450 msgr2=0x7fd6b0198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ff450 0x7fd6b0198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 -- 172.21.15.86:0/2085027870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0095f0 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6afe00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ff450 0x7fd6b0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.358+0000 7fd6af400700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0199220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006010 tx=0x7fd69c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.359+0000 7fd6acc00700 1 -- 172.21.15.86:0/2085027870 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c01d070 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.359+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b01a0db0 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.359+0000 7fd6acc00700 1 -- 172.21.15.86:0/2085027870 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd69c005030 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.359+0000 7fd6acc00700 1 -- 172.21.15.86:0/2085027870 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c004bb0 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.359+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01a11d0 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.363+0000 7fd6acc00700 1 -- 172.21.15.86:0/2085027870 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fd69c004d10 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.364+0000 7fd6cabb1b80 1 Processor -- start 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.364+0000 7fd6cabb1b80 1 -- start start 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3aafa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ab7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c39ff990 con 0x5625c3ab11f0 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c39fd350 con 0x5625c3aaf5f0 2024-03-22T19:11:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c39f8c40 con 0x5625c3ab0830 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ab7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6b5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3aafa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ab7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55048/0 (socket says 172.21.15.86:55048) 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.365+0000 7fd6af400700 1 -- 172.21.15.86:0/2397682454 learned_addr learned my addr 172.21.15.86:0/2397682454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6af400700 1 -- 172.21.15.86:0/2397682454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 msgr2=0x5625c3ab0cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6af400700 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab0cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6af400700 1 -- 172.21.15.86:0/2397682454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 msgr2=0x5625c3aafa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6af400700 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3aafa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6af400700 1 -- 172.21.15.86:0/2397682454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625c387a3b0 con 0x5625c3ab11f0 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6b5000700 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3aafa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6af400700 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ab7a80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0d0c90 tx=0x7fd69c0d5370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6af7d627bfdf5d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.366+0000 7fd6acc00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd6940a0b10 0x7fd6940a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.367+0000 7fd6a5600700 1 -- 172.21.15.86:0/2397682454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c096780 con 0x5625c3ab11f0 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.367+0000 7fd6a5600700 1 -- 172.21.15.86:0/2397682454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd69c096050 con 0x5625c3ab11f0 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.367+0000 7fd6a5600700 1 -- 172.21.15.86:0/2397682454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c09bb10 con 0x5625c3ab11f0 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.367+0000 7fd6acc00700 1 -- 172.21.15.86:0/2085027870 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fd69c0ccd30 con 0x7fd6b00ffdb0 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.367+0000 7fd6b5000700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd6940a0b10 0x7fd6940a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2397682454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 msgr2=0x5625c3ab7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ab7a80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0d0c90 tx=0x7fd69c0d5370 comp rx=0 tx=0).stop 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6b5000700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd6940a0b10 0x7fd6940a2fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5625c3aaf390 tx=0x7fd6a800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2397682454 shutdown_connections 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ab7a80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3aafa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2397682454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2397682454 >> 172.21.15.86:0/2397682454 conn(0x5625c3a1c220 msgr2=0x5625c3aa1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2397682454 shutdown_connections 2024-03-22T19:11:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.368+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2397682454 wait complete. 2024-03-22T19:11:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.369+0000 7fd6cabb1b80 1 -- start start 2024-03-22T19:11:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.369+0000 7fd6cabb1b80 1 Processor -- start 2024-03-22T19:11:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ac06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3994d80 con 0x5625c3ab11f0 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3994a20 con 0x5625c3aaf5f0 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c387c6a0 con 0x5625c3ab0830 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6cabb1b80 10 client.0 _ll_register_callbacks cb 0x5625c38003a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6af400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6af400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54882/0 (socket says 172.21.15.86:54882) 2024-03-22T19:11:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6af400700 1 -- 172.21.15.86:0/2456025586 learned_addr learned my addr 172.21.15.86:0/2456025586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6b5000700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ac06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.370+0000 7fd6afe00700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6af400700 1 -- 172.21.15.86:0/2456025586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 msgr2=0x5625c3ab9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6af400700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6af400700 1 -- 172.21.15.86:0/2456025586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 msgr2=0x5625c3ac06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6af400700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ac06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6af400700 1 -- 172.21.15.86:0/2456025586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a8011230 con 0x5625c3ab0830 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6afe00700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6af400700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab9680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0d0c60 tx=0x7fd69c00b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb7877cc7623a7d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd69a000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd69a000700 1 -- 172.21.15.86:0/2456025586 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c09e0b0 con 0x5625c3ab0830 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd69a000700 1 -- 172.21.15.86:0/2456025586 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fd69c0a0040 con 0x5625c3ab0830 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.371+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625c3a99c30 con 0x5625c3ab0830 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.372+0000 7fd69a000700 1 -- 172.21.15.86:0/2456025586 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd69c09ab40 con 0x5625c3ab0830 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.372+0000 7fd6cabb1b80 10 client.25609 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.372+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd6b01a11d0 con 0x5625c3ab0830 2024-03-22T19:11:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.372+0000 7fd69a000700 1 -- 172.21.15.86:0/2456025586 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625c3a99c30 con 0x5625c3ab0830 2024-03-22T19:11:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd69a000700 1 -- 172.21.15.86:0/2456025586 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd69c005480 con 0x5625c3ab0830 2024-03-22T19:11:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6cabb1b80 10 client.25609 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6cabb1b80 10 client.25609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6cabb1b80 20 client.25609 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3ac49b0 0x5625c3ac6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6cabb1b80 4 client.25609 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625c39f1700 con 0x5625c3ac49b0 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.373+0000 7fd6afe00700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3ac49b0 0x5625c3ac6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.374+0000 7fd69a000700 10 client.25609 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.374+0000 7fd6afe00700 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3ac49b0 0x5625c3ac6dd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccf38993ddcf604 server_cookie=2dbf9d41c6dcaba8 in_seq=0 out_seq=0 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.374+0000 7fd69a000700 1 -- 172.21.15.86:0/2456025586 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5625c39f1700 con 0x5625c3ac49b0 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.374+0000 7fd69a000700 10 client.25609 handle_command_reply: tid=0 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 client.25609 shutdown 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 msgr2=0x5625c3ab9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab9680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0d0c60 tx=0x7fd69c00b540 comp rx=0 tx=0).stop 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 shutdown_connections 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3ac49b0 0x5625c3ac6dd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3ac06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/2456025586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ab9680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 >> 172.21.15.86:0/2456025586 conn(0x5625c3a1c220 msgr2=0x5625c3aa1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd69a000700 0 client.25609 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 shutdown_connections 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/2456025586 wait complete. 2024-03-22T19:11:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.375+0000 7fd6cabb1b80 20 client.25609 trim_cache size 0 max 16384 2024-03-22T19:11:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 Processor -- start 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 -- start start 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ac0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3ab11f0 0x5625c3ab8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c39f1730 con 0x5625c3aaf5f0 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3a9e770 con 0x5625c3ab11f0 2024-03-22T19:11:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.389+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3a9c130 con 0x5625c3ab0830 2024-03-22T19:11:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ac0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6b5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3ab11f0 0x5625c3ab8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ac0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54888/0 (socket says 172.21.15.86:54888) 2024-03-22T19:11:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 -- 172.21.15.86:0/1302200636 learned_addr learned my addr 172.21.15.86:0/1302200636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6afe00700 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55058/0 (socket says 172.21.15.86:55058) 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 -- 172.21.15.86:0/1302200636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3ab11f0 msgr2=0x5625c3ab8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3ab11f0 0x5625c3ab8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 -- 172.21.15.86:0/1302200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 msgr2=0x5625c3ab0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6af400700 1 -- 172.21.15.86:0/1302200636 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625c3826b60 con 0x5625c3ab0830 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.390+0000 7fd6afe00700 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.391+0000 7fd6af400700 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ac0680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0d0c90 tx=0x7fd69c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3745c545ae1e943b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.391+0000 7fd6a6000700 1 -- 172.21.15.86:0/1302200636 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c099da0 con 0x5625c3ab0830 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.391+0000 7fd6a6000700 1 -- 172.21.15.86:0/1302200636 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd69c0cf030 con 0x5625c3ab0830 2024-03-22T19:11:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.391+0000 7fd6a6000700 1 -- 172.21.15.86:0/1302200636 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c09a6a0 con 0x5625c3ab0830 2024-03-22T19:11:58.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/1302200636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 msgr2=0x5625c3ac0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ac0680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0d0c90 tx=0x7fd69c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/1302200636 shutdown_connections 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3ab11f0 0x5625c3ab8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3aaf5f0 0x5625c3ab0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/1302200636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3ac0680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/1302200636 >> 172.21.15.86:0/1302200636 conn(0x5625c3a1c220 msgr2=0x5625c3ab4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/1302200636 shutdown_connections 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.392+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/1302200636 wait complete. 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 -- start start 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 Processor -- start 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3abe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3abe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3abf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3abfce0 con 0x5625c3ab11f0 2024-03-22T19:11:58.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3abfe50 con 0x5625c3aaf5f0 2024-03-22T19:11:58.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625c3ac0010 con 0x5625c3ab0830 2024-03-22T19:11:58.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.393+0000 7fd6cabb1b80 10 client.0 _ll_register_callbacks cb 0x7fd6a8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3abe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6b5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3abf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3abe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:45060/0 (socket says 172.21.15.86:45060) 2024-03-22T19:11:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 -- 172.21.15.86:0/3422498922 learned_addr learned my addr 172.21.15.86:0/3422498922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6af400700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3abe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 -- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 msgr2=0x5625c3abe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3abe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 -- 172.21.15.86:0/3422498922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 msgr2=0x5625c3abf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3abf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.394+0000 7fd6afe00700 1 -- 172.21.15.86:0/3422498922 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01a11d0 con 0x5625c3aaf5f0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.395+0000 7fd6af400700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3abe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.395+0000 7fd6afe00700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3abe390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5625c3ac7670 tx=0x7fd6a0004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=187df2e887b58b7b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.395+0000 7fd69a000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.395+0000 7fd69a000700 1 -- 172.21.15.86:0/3422498922 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0018040 con 0x5625c3aaf5f0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.395+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd69c096050 con 0x5625c3aaf5f0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.396+0000 7fd69a000700 1 -- 172.21.15.86:0/3422498922 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fd6a001a890 con 0x5625c3aaf5f0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.396+0000 7fd69a000700 1 -- 172.21.15.86:0/3422498922 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a001aa50 con 0x5625c3aaf5f0 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.396+0000 7fd6cabb1b80 10 client.25619 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.396+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625c39beea0 con 0x5625c3aaf5f0 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.396+0000 7fd69a000700 1 -- 172.21.15.86:0/3422498922 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd69c096050 con 0x5625c3aaf5f0 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd69a000700 1 -- 172.21.15.86:0/3422498922 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd6a00135e0 con 0x5625c3aaf5f0 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6cabb1b80 10 client.25619 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6cabb1b80 10 client.25619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6cabb1b80 20 client.25619 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3b17350 0x5625c3b19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6cabb1b80 4 client.25619 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5625c3847cf0 con 0x5625c3b17350 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.397+0000 7fd6af400700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3b17350 0x5625c3b19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6af400700 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3b17350 0x5625c3b19770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8332031d040f0f80 server_cookie=822fd7061cde8ad1 in_seq=0 out_seq=0 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd69a000700 10 client.25619 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd69a000700 1 -- 172.21.15.86:0/3422498922 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5625c3847cf0 con 0x5625c3b17350 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd69a000700 10 client.25619 handle_command_reply: tid=0 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 client.25619 shutdown 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 msgr2=0x5625c3abe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3abe390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5625c3ac7670 tx=0x7fd6a0004320 comp rx=0 tx=0).stop 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 shutdown_connections 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5625c3b17350 0x5625c3b19770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625c3ab11f0 0x5625c3abf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5625c3aaf5f0 0x5625c3abe390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 --2- 172.21.15.86:0/3422498922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625c3ab0830 0x5625c3abe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd69a000700 0 client.25619 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:11:58.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.398+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 >> 172.21.15.86:0/3422498922 conn(0x5625c3a1c220 msgr2=0x5625c3ab7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 shutdown_connections 2024-03-22T19:11:58.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6cabb1b80 1 -- 172.21.15.86:0/3422498922 wait complete. 2024-03-22T19:11:58.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6cabb1b80 20 client.25619 trim_cache size 0 max 16384 2024-03-22T19:11:58.402 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd6940a0b10 msgr2=0x7fd6940a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd6940a0b10 0x7fd6940a2fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5625c3aaf390 tx=0x7fd6a800b040 comp rx=0 tx=0).stop 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 msgr2=0x7fd6b0199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.399+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0199220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006010 tx=0x7fd69c00f690 comp rx=0 tx=0).stop 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 shutdown_connections 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fd6940a0b10 0x7fd6940a2fd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6b0100770 0x7fd6b01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b00ff450 0x7fd6b0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 --2- 172.21.15.86:0/2085027870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b00ffdb0 0x7fd6b0199220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 >> 172.21.15.86:0/2085027870 conn(0x7fd6b00fafb0 msgr2=0x7fd6b00fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 shutdown_connections 2024-03-22T19:11:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.400+0000 7fd6b5a00700 1 -- 172.21.15.86:0/2085027870 wait complete. 2024-03-22T19:11:58.419 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:58 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2408474117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:11:58.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:11:58.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:58 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2408474117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:11:58.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:58 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2408474117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:11:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 -- 172.21.15.86:0/2102318766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 msgr2=0x7ff8b4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2102318766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4100230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00099c0 tx=0x7ff8a0009cd0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 -- 172.21.15.86:0/2102318766 shutdown_connections 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2102318766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b4100770 0x7ff8b4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2102318766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40ff450 0x7ff8b40ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2102318766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4100230 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 -- 172.21.15.86:0/2102318766 >> 172.21.15.86:0/2102318766 conn(0x7ff8b40fafb0 msgr2=0x7ff8b40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 -- 172.21.15.86:0/2102318766 shutdown_connections 2024-03-22T19:11:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.631+0000 7ff8bae00700 1 -- 172.21.15.86:0/2102318766 wait complete. 2024-03-22T19:11:58.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 Processor -- start 2024-03-22T19:11:58.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 -- start start 2024-03-22T19:11:58.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 0x7ff8b4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4100770 0x7ff8b41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b41a0800 con 0x7ff8b40ff450 2024-03-22T19:11:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b41a0970 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.633+0000 7ff8bae00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b41a0b30 con 0x7ff8b4100770 2024-03-22T19:11:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 0x7ff8b4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 0x7ff8b4198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36052/0 (socket says 172.21.15.86:36052) 2024-03-22T19:11:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9a00700 1 -- 172.21.15.86:0/2638046497 learned_addr learned my addr 172.21.15.86:0/2638046497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8ba400700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4100770 0x7ff8b41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9000700 1 -- 172.21.15.86:0/2638046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4100770 msgr2=0x7ff8b41a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9000700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4100770 0x7ff8b41a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9000700 1 -- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 msgr2=0x7ff8b4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9000700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 0x7ff8b4198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.634+0000 7ff8b9000700 1 -- 172.21.15.86:0/2638046497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a00095f0 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 0x7ff8b4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8b9000700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4199220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00098e0 tx=0x7ff8a000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8b2000700 1 -- 172.21.15.86:0/2638046497 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001d070 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8b2000700 1 -- 172.21.15.86:0/2638046497 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a0005020 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8b2000700 1 -- 172.21.15.86:0/2638046497 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00049e0 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b41a0db0 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.635+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b41a1120 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.640+0000 7ff8b2000700 1 -- 172.21.15.86:0/2638046497 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff8a0005190 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 Processor -- start 2024-03-22T19:11:58.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 -- start start 2024-03-22T19:11:58.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7cde0 0x5567d7f7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f7e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7ed5990 con 0x5567d7f7e9e0 2024-03-22T19:11:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7ed3350 con 0x5567d7f7cde0 2024-03-22T19:11:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.641+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7ecec40 con 0x5567d7f7e020 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36058/0 (socket says 172.21.15.86:36058) 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 -- 172.21.15.86:0/1488344500 learned_addr learned my addr 172.21.15.86:0/1488344500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8ba400700 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7cde0 0x5567d7f7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b2000700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff89c0a0b10 0x7ff89c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 -- 172.21.15.86:0/1488344500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 msgr2=0x5567d7f7e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f7e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 -- 172.21.15.86:0/1488344500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7cde0 msgr2=0x5567d7f7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7cde0 0x5567d7f7d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.642+0000 7ff8b9000700 1 -- 172.21.15.86:0/1488344500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d7d503b0 con 0x5567d7f7e9e0 2024-03-22T19:11:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.643+0000 7ff8ba400700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff89c0a0b10 0x7ff89c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.643+0000 7ff8b2000700 1 -- 172.21.15.86:0/2638046497 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff8a00cc990 con 0x7ff8b40ffdb0 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.643+0000 7ff8b9000700 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00c8270 tx=0x7ff8a00c4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcead51286ed81f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.643+0000 7ff8a6a00700 1 -- 172.21.15.86:0/1488344500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00d2590 con 0x5567d7f7e9e0 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.643+0000 7ff8a6a00700 1 -- 172.21.15.86:0/1488344500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a0099e70 con 0x5567d7f7e9e0 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.643+0000 7ff8a6a00700 1 -- 172.21.15.86:0/1488344500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a006d9c0 con 0x5567d7f7e9e0 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1488344500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 msgr2=0x5567d7f85270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00c8270 tx=0x7ff8a00c4550 comp rx=0 tx=0).stop 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8ba400700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff89c0a0b10 0x7ff89c0a2fd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5567d7f7cb80 tx=0x7ff8ac009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1488344500 shutdown_connections 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85270 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7cde0 0x5567d7f7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1488344500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1488344500 >> 172.21.15.86:0/1488344500 conn(0x5567d7ef2220 msgr2=0x5567d7f77e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.644+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1488344500 shutdown_connections 2024-03-22T19:11:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.645+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1488344500 wait complete. 2024-03-22T19:11:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.646+0000 7ff8cffc3b80 1 -- start start 2024-03-22T19:11:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.646+0000 7ff8cffc3b80 1 Processor -- start 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 0x5567d7f868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f86e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7f8de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7e6ad80 con 0x5567d7f7cde0 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7e6aa20 con 0x5567d7f7e9e0 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7d526a0 con 0x5567d7f7e020 2024-03-22T19:11:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8cffc3b80 10 client.0 _ll_register_callbacks cb 0x5567d7cd63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 0x5567d7f868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 0x5567d7f868e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36060/0 (socket says 172.21.15.86:36060) 2024-03-22T19:11:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8b9a00700 1 -- 172.21.15.86:0/4039913083 learned_addr learned my addr 172.21.15.86:0/4039913083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8b9000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f86e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.647+0000 7ff8ba400700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7f8de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9a00700 1 -- 172.21.15.86:0/4039913083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 msgr2=0x5567d7f86e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f86e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9a00700 1 -- 172.21.15.86:0/4039913083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 msgr2=0x5567d7f8de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7f8de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9a00700 1 -- 172.21.15.86:0/4039913083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a00d5040 con 0x5567d7f7cde0 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9000700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 0x5567d7f868e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8011d00 tx=0x7ff8a8011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb58878c74f69fe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.648+0000 7ff89b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.649+0000 7ff89b400700 1 -- 172.21.15.86:0/4039913083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800fd00 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.649+0000 7ff89b400700 1 -- 172.21.15.86:0/4039913083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff8a800fe60 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.649+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567d7f6fc30 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.649+0000 7ff89b400700 1 -- 172.21.15.86:0/4039913083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a8019430 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.649+0000 7ff8cffc3b80 10 client.15612 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.649+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567d7cfcb60 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff89b400700 1 -- 172.21.15.86:0/4039913083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a8028080 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff89b400700 1 -- 172.21.15.86:0/4039913083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff8a8023540 con 0x5567d7f7cde0 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff8cffc3b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff8cffc3b80 10 client.15612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff8cffc3b80 20 client.15612 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7f926d0 0x5567d7f94af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff8cffc3b80 4 client.15612 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.650+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567d7ec7700 con 0x5567d7f926d0 2024-03-22T19:11:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.651+0000 7ff8b9000700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7f926d0 0x5567d7f94af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.651+0000 7ff8b9000700 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7f926d0 0x5567d7f94af0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11d8b0940aff1abd server_cookie=f0ebad9e3dfe21d in_seq=0 out_seq=0 2024-03-22T19:11:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.651+0000 7ff89b400700 10 client.15612 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.652+0000 7ff89b400700 1 -- 172.21.15.86:0/4039913083 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5567d7ec7700 con 0x5567d7f926d0 2024-03-22T19:11:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.652+0000 7ff89b400700 10 client.15612 handle_command_reply: tid=0 2024-03-22T19:11:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.652+0000 7ff8cffc3b80 1 client.15612 shutdown 2024-03-22T19:11:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 msgr2=0x5567d7f868e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 0x5567d7f868e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8011d00 tx=0x7ff8a8011d30 comp rx=0 tx=0).stop 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 shutdown_connections 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7f926d0 0x5567d7f94af0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7f8de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7cde0 0x5567d7f868e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/4039913083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7f86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 >> 172.21.15.86:0/4039913083 conn(0x5567d7ef2220 msgr2=0x5567d7f77350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff89b400700 0 client.15612 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 shutdown_connections 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.655+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/4039913083 wait complete. 2024-03-22T19:11:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.656+0000 7ff8cffc3b80 20 client.15612 trim_cache size 0 max 16384 2024-03-22T19:11:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 Processor -- start 2024-03-22T19:11:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 -- start start 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 0x5567d7f8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f81810 0x5567d7f81c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7f821d0 con 0x5567d7f7e020 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7f8e8d0 con 0x5567d7f81810 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.669+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7f823a0 con 0x5567d7f7e9e0 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 0x5567d7f8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 0x5567d7f8be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36062/0 (socket says 172.21.15.86:36062) 2024-03-22T19:11:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 -- 172.21.15.86:0/1353902239 learned_addr learned my addr 172.21.15.86:0/1353902239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8ba400700 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f81810 0x5567d7f81c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9000700 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 -- 172.21.15.86:0/1353902239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e9e0 msgr2=0x5567d7f85e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 -- 172.21.15.86:0/1353902239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f81810 msgr2=0x5567d7f81c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f81810 0x5567d7f81c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 -- 172.21.15.86:0/1353902239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d7e94ea0 con 0x5567d7f7e020 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.670+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 0x5567d7f8be10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8023920 tx=0x7ff8a8009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6ffee6f1d095ae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.671+0000 7ff8a7400700 1 -- 172.21.15.86:0/1353902239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8023e50 con 0x5567d7f7e020 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.671+0000 7ff8a7400700 1 -- 172.21.15.86:0/1353902239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567d7f6fc30 con 0x5567d7f7e020 2024-03-22T19:11:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.671+0000 7ff8a7400700 1 -- 172.21.15.86:0/1353902239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80054e0 con 0x5567d7f7e020 2024-03-22T19:11:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.671+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1353902239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 msgr2=0x5567d7f8be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 0x5567d7f8be10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8023920 tx=0x7ff8a8009350 comp rx=0 tx=0).stop 2024-03-22T19:11:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1353902239 shutdown_connections 2024-03-22T19:11:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f81810 0x5567d7f81c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f7e020 0x5567d7f8be10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/1353902239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e9e0 0x5567d7f85e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1353902239 >> 172.21.15.86:0/1353902239 conn(0x5567d7ef2220 msgr2=0x5567d7f73b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1353902239 shutdown_connections 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/1353902239 wait complete. 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 -- start start 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.672+0000 7ff8cffc3b80 1 Processor -- start 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7fe6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7fe9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f81810 0x5567d7feb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7febe80 con 0x5567d7f81810 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7febff0 con 0x5567d7f7e9e0 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d7fec1b0 con 0x5567d7f7e020 2024-03-22T19:11:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8cffc3b80 10 client.0 _ll_register_callbacks cb 0x7ff8a80117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8ba400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f81810 0x5567d7feb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7fe6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8b9a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7fe6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43026/0 (socket says 172.21.15.86:43026) 2024-03-22T19:11:58.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.673+0000 7ff8b9a00700 1 -- 172.21.15.86:0/862380159 learned_addr learned my addr 172.21.15.86:0/862380159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8b9000700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7fe9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8ba400700 1 -- 172.21.15.86:0/862380159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 msgr2=0x5567d7fe6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8ba400700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7fe6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8ba400700 1 -- 172.21.15.86:0/862380159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 msgr2=0x5567d7fe9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8ba400700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7fe9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8ba400700 1 -- 172.21.15.86:0/862380159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567d7cfcb60 con 0x5567d7f81810 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8b9000700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7fe9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff8ba400700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f81810 0x5567d7feb770 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac024620 tx=0x7ff8ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80796cb745bd6139 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.674+0000 7ff89b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.675+0000 7ff89b400700 1 -- 172.21.15.86:0/862380159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00e040 con 0x5567d7f81810 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.675+0000 7ff89b400700 1 -- 172.21.15.86:0/862380159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff8ac01c040 con 0x5567d7f81810 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.675+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8a0099e70 con 0x5567d7f81810 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.675+0000 7ff89b400700 1 -- 172.21.15.86:0/862380159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8ac017e90 con 0x5567d7f81810 2024-03-22T19:11:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.675+0000 7ff8cffc3b80 10 client.15618 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.675+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567d7d54cd0 con 0x5567d7f81810 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff89b400700 1 -- 172.21.15.86:0/862380159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a0099e70 con 0x5567d7f81810 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff89b400700 1 -- 172.21.15.86:0/862380159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff8ac02e530 con 0x5567d7f81810 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8cffc3b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8cffc3b80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8cffc3b80 20 client.15618 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7fedd10 0x5567d7ff0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8cffc3b80 4 client.15618 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5567d7d1dcf0 con 0x5567d7fedd10 2024-03-22T19:11:58.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.676+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7fedd10 0x5567d7ff0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.677+0000 7ff89b400700 10 client.15618 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:58.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.677+0000 7ff8b9a00700 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7fedd10 0x5567d7ff0130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81f3a3e83d4f69e7 server_cookie=b69958b940bd5a9d in_seq=0 out_seq=0 2024-03-22T19:11:58.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff89b400700 1 -- 172.21.15.86:0/862380159 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5567d7d1dcf0 con 0x5567d7fedd10 2024-03-22T19:11:58.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff89b400700 10 client.15618 handle_command_reply: tid=0 2024-03-22T19:11:58.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 client.15618 shutdown 2024-03-22T19:11:58.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f81810 msgr2=0x5567d7feb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f81810 0x5567d7feb770 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac024620 tx=0x7ff8ac004ad0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 shutdown_connections 2024-03-22T19:11:58.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5567d7fedd10 0x5567d7ff0130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d7f81810 0x5567d7feb770 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567d7f7e020 0x5567d7fe6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 --2- 172.21.15.86:0/862380159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567d7f7e9e0 0x5567d7fe9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 >> 172.21.15.86:0/862380159 conn(0x5567d7ef2220 msgr2=0x5567d7f86690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff89b400700 0 client.15618 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 shutdown_connections 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.679+0000 7ff8cffc3b80 1 -- 172.21.15.86:0/862380159 wait complete. 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.680+0000 7ff8cffc3b80 20 client.15618 trim_cache size 0 max 16384 2024-03-22T19:11:58.682 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.680+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff89c0a0b10 msgr2=0x7ff89c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.680+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff89c0a0b10 0x7ff89c0a2fd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5567d7f7cb80 tx=0x7ff8ac009040 comp rx=0 tx=0).stop 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.680+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 msgr2=0x7ff8b4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.680+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4199220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00098e0 tx=0x7ff8a000f690 comp rx=0 tx=0).stop 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.680+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 shutdown_connections 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff89c0a0b10 0x7ff89c0a2fd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4100770 0x7ff8b41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8b40ff450 0x7ff8b4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 --2- 172.21.15.86:0/2638046497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8b40ffdb0 0x7ff8b4199220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 >> 172.21.15.86:0/2638046497 conn(0x7ff8b40fafb0 msgr2=0x7ff8b40fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 shutdown_connections 2024-03-22T19:11:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.681+0000 7ff8bae00700 1 -- 172.21.15.86:0/2638046497 wait complete. 2024-03-22T19:11:58.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:11:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.948+0000 7f13e0e00700 1 -- 172.21.15.86:0/757429409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc0ffdb0 msgr2=0x7f13dc100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.948+0000 7f13e0e00700 1 --2- 172.21.15.86:0/757429409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc100230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009920 tx=0x7f13c8009c30 comp rx=0 tx=0).stop 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 -- 172.21.15.86:0/757429409 shutdown_connections 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 --2- 172.21.15.86:0/757429409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc100770 0x7f13dc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 --2- 172.21.15.86:0/757429409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 0x7f13dc0ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 --2- 172.21.15.86:0/757429409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc100230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 -- 172.21.15.86:0/757429409 >> 172.21.15.86:0/757429409 conn(0x7f13dc0fafb0 msgr2=0x7f13dc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 -- 172.21.15.86:0/757429409 shutdown_connections 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.949+0000 7f13e0e00700 1 -- 172.21.15.86:0/757429409 wait complete. 2024-03-22T19:11:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.950+0000 7f13e0e00700 1 Processor -- start 2024-03-22T19:11:58.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 -- start start 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 0x7f13dc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc100770 0x7f13dc1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1a0840 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1a09b0 con 0x7f13dc100770 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13e0e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1a0b70 con 0x7f13dc0ff450 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13daa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13daa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36094/0 (socket says 172.21.15.86:36094) 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13daa00700 1 -- 172.21.15.86:0/2300782590 learned_addr learned my addr 172.21.15.86:0/2300782590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.951+0000 7f13db400700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 0x7f13dc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13dbe00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc100770 0x7f13dc1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13daa00700 1 -- 172.21.15.86:0/2300782590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 msgr2=0x7f13dc198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13daa00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 0x7f13dc198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13daa00700 1 -- 172.21.15.86:0/2300782590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc100770 msgr2=0x7f13dc1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13daa00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc100770 0x7f13dc1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13daa00700 1 -- 172.21.15.86:0/2300782590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8009520 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13db400700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 0x7f13dc198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.952+0000 7f13daa00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc199290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009230 tx=0x7f13c8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.953+0000 7f13d3e00700 1 -- 172.21.15.86:0/2300782590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c801c070 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.953+0000 7f13d3e00700 1 -- 172.21.15.86:0/2300782590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c800bc50 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.953+0000 7f13d3e00700 1 -- 172.21.15.86:0/2300782590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8004b10 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.953+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc1a0df0 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.953+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc1a11c0 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.956+0000 7f13d3e00700 1 -- 172.21.15.86:0/2300782590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f13c8020070 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.958+0000 7f13d3e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13c00a0b10 0x7f13c00a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.958+0000 7f13d3e00700 1 -- 172.21.15.86:0/2300782590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f13c80cce60 con 0x7f13dc0ffdb0 2024-03-22T19:11:58.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.958+0000 7f13db400700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13c00a0b10 0x7f13c00a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.959+0000 7f13f5fe4b80 1 Processor -- start 2024-03-22T19:11:58.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.959+0000 7f13db400700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13c00a0b10 0x7f13c00a2fd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f13dc19a1d0 tx=0x7f13cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.959+0000 7f13f5fe4b80 1 -- start start 2024-03-22T19:11:58.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x56501220f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 0x5650122104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x565012217270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x565012167990 con 0x56501220ede0 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565012165350 con 0x5650122109e0 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x565012160c40 con 0x565012210020 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13daa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 0x5650122104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13daa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 0x5650122104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43062/0 (socket says 172.21.15.86:43062) 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13daa00700 1 -- 172.21.15.86:0/906341827 learned_addr learned my addr 172.21.15.86:0/906341827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13dbe00700 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x565012217270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13db400700 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x56501220f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13daa00700 1 -- 172.21.15.86:0/906341827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 msgr2=0x565012217270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13daa00700 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x565012217270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.960+0000 7f13daa00700 1 -- 172.21.15.86:0/906341827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 msgr2=0x56501220f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13daa00700 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x56501220f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13daa00700 1 -- 172.21.15.86:0/906341827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565011fe23b0 con 0x565012210020 2024-03-22T19:11:58.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13db400700 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x56501220f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:11:58.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13daa00700 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 0x5650122104a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13c8096a70 tx=0x7f13c8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29325b78ad9259e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13d0c00700 1 -- 172.21.15.86:0/906341827 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c809e3a0 con 0x565012210020 2024-03-22T19:11:58.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13d0c00700 1 -- 172.21.15.86:0/906341827 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c80a2960 con 0x565012210020 2024-03-22T19:11:58.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.961+0000 7f13d0c00700 1 -- 172.21.15.86:0/906341827 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8004e70 con 0x565012210020 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/906341827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 msgr2=0x5650122104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 0x5650122104a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13c8096a70 tx=0x7f13c8005e20 comp rx=0 tx=0).stop 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/906341827 shutdown_connections 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x565012217270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x56501220f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/906341827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565012210020 0x5650122104a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/906341827 >> 172.21.15.86:0/906341827 conn(0x565012184220 msgr2=0x565012209e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.962+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/906341827 shutdown_connections 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.963+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/906341827 wait complete. 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.963+0000 7f13f5fe4b80 1 -- start start 2024-03-22T19:11:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.963+0000 7f13f5fe4b80 1 Processor -- start 2024-03-22T19:11:58.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x5650122188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x565012218df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650122109e0 0x56501221fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650120fcd80 con 0x56501220ede0 2024-03-22T19:11:58.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650120fca20 con 0x565012210020 2024-03-22T19:11:58.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x565011fe46a0 con 0x5650122109e0 2024-03-22T19:11:58.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.964+0000 7f13f5fe4b80 10 client.0 _ll_register_callbacks cb 0x565011f683a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x5650122188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x565012218df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x5650122188b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36102/0 (socket says 172.21.15.86:36102) 2024-03-22T19:11:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13db400700 1 -- 172.21.15.86:0/2023690031 learned_addr learned my addr 172.21.15.86:0/2023690031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13dbe00700 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650122109e0 0x56501221fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 -- 172.21.15.86:0/2023690031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650122109e0 msgr2=0x56501221fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650122109e0 0x56501221fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 -- 172.21.15.86:0/2023690031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 msgr2=0x5650122188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x5650122188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 -- 172.21.15.86:0/2023690031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc1a11c0 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13daa00700 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x565012218df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13c809a500 tx=0x7f13c8098b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc1d9bf687620d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.965+0000 7f13c5600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13c5600700 1 -- 172.21.15.86:0/2023690031 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80b5060 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13c5600700 1 -- 172.21.15.86:0/2023690031 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f13c809e170 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565012201c30 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13c5600700 1 -- 172.21.15.86:0/2023690031 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c809e3a0 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13f5fe4b80 10 client.25649 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565011f8eb60 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13c5600700 1 -- 172.21.15.86:0/2023690031 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565012201c30 con 0x565012210020 2024-03-22T19:11:58.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.966+0000 7f13c5600700 1 -- 172.21.15.86:0/2023690031 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f13c8096360 con 0x565012210020 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13f5fe4b80 10 client.25649 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13f5fe4b80 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13f5fe4b80 20 client.25649 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x565012224210 0x565012226630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13f5fe4b80 4 client.25649 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565012159700 con 0x565012224210 2024-03-22T19:11:58.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.967+0000 7f13db400700 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x565012224210 0x565012226630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.968+0000 7f13db400700 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x565012224210 0x565012226630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea61dc79d42b760 server_cookie=61d49a741280732 in_seq=0 out_seq=0 2024-03-22T19:11:58.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.968+0000 7f13c5600700 10 client.25649 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:58.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.968+0000 7f13c5600700 1 -- 172.21.15.86:0/2023690031 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x565012159700 con 0x565012224210 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.968+0000 7f13c5600700 10 client.25649 handle_command_reply: tid=0 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.968+0000 7f13f5fe4b80 1 client.25649 shutdown 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 msgr2=0x565012218df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x565012218df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13c809a500 tx=0x7f13c8098b20 comp rx=0 tx=0).stop 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 shutdown_connections 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x565012224210 0x565012226630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650122109e0 0x56501221fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56501220ede0 0x5650122188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2023690031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x565012218df0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 >> 172.21.15.86:0/2023690031 conn(0x565012184220 msgr2=0x565012209350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13c5600700 0 client.25649 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 shutdown_connections 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2023690031 wait complete. 2024-03-22T19:11:58.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.969+0000 7f13f5fe4b80 20 client.25649 trim_cache size 0 max 16384 2024-03-22T19:11:58.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 Processor -- start 2024-03-22T19:11:58.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 -- start start 2024-03-22T19:11:58.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x565012217dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x56501221fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650122109e0 0x565012209350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501220ea80 con 0x5650122109e0 2024-03-22T19:11:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565012206740 con 0x565012210020 2024-03-22T19:11:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.983+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501220f6c0 con 0x56501220ede0 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13db400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x565012217dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650122109e0 0x565012209350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13db400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x565012217dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43078/0 (socket says 172.21.15.86:43078) 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13db400700 1 -- 172.21.15.86:0/2830721181 learned_addr learned my addr 172.21.15.86:0/2830721181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13daa00700 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x56501221fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 -- 172.21.15.86:0/2830721181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 msgr2=0x565012217dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x565012217dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 -- 172.21.15.86:0/2830721181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 msgr2=0x56501221fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x56501221fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 -- 172.21.15.86:0/2830721181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565012126ea0 con 0x5650122109e0 2024-03-22T19:11:58.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13daa00700 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x56501221fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:11:58.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.984+0000 7f13dbe00700 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650122109e0 0x565012209350 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009cf0 tx=0x7f13d4009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6061e703f5b25ec server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.985+0000 7f13d1600700 1 -- 172.21.15.86:0/2830721181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4004bb0 con 0x5650122109e0 2024-03-22T19:11:58.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.985+0000 7f13d1600700 1 -- 172.21.15.86:0/2830721181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d400f040 con 0x5650122109e0 2024-03-22T19:11:58.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.985+0000 7f13d1600700 1 -- 172.21.15.86:0/2830721181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4017d40 con 0x5650122109e0 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2830721181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650122109e0 msgr2=0x565012209350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650122109e0 0x565012209350 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009cf0 tx=0x7f13d4009130 comp rx=0 tx=0).stop 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2830721181 shutdown_connections 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650122109e0 0x565012209350 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x565012217dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/2830721181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565012210020 0x56501221fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2830721181 >> 172.21.15.86:0/2830721181 conn(0x565012184220 msgr2=0x565012226220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2830721181 shutdown_connections 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/2830721181 wait complete. 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.986+0000 7f13f5fe4b80 1 -- start start 2024-03-22T19:11:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 Processor -- start 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x56501221dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565012210020 0x56501221e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x56501221f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501221f7b0 con 0x565012210020 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501221f920 con 0x5650122109e0 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501221fae0 con 0x56501220ede0 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.987+0000 7f13f5fe4b80 10 client.0 _ll_register_callbacks cb 0x7f13c8099ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13db400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x56501221dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13daa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565012210020 0x56501221e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13db400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x56501221dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43090/0 (socket says 172.21.15.86:43090) 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13db400700 1 -- 172.21.15.86:0/3226807484 learned_addr learned my addr 172.21.15.86:0/3226807484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13dbe00700 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x56501221f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13daa00700 1 -- 172.21.15.86:0/3226807484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 msgr2=0x56501221dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.988+0000 7f13daa00700 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x56501221dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13daa00700 1 -- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 msgr2=0x56501221f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:11:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13daa00700 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x56501221f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13daa00700 1 -- 172.21.15.86:0/3226807484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565011f8eb60 con 0x565012210020 2024-03-22T19:11:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13daa00700 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565012210020 0x56501221e2b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f13c80966f0 tx=0x7f13c8099f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f90ed574fc39 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:11:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13c5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:11:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13c5600700 1 -- 172.21.15.86:0/3226807484 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c809e170 con 0x565012210020 2024-03-22T19:11:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13c5600700 1 -- 172.21.15.86:0/3226807484 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f13c809e3a0 con 0x565012210020 2024-03-22T19:11:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13c80a2960 con 0x565012210020 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13c5600700 1 -- 172.21.15.86:0/3226807484 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c80053c0 con 0x565012210020 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13f5fe4b80 10 client.15633 fetch_fsmap learned FSMap version 27 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.989+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f13d40099d0 con 0x565012210020 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.990+0000 7f13c5600700 1 -- 172.21.15.86:0/3226807484 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c80a2960 con 0x565012210020 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13c5600700 1 -- 172.21.15.86:0/3226807484 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f13c8082080 con 0x565012210020 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13f5fe4b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13f5fe4b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13f5fe4b80 20 client.15633 populate_metadata read hostname 'smithi086' 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5650122768a0 0x565012278cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13f5fe4b80 4 client.15633 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565011fafcf0 con 0x5650122768a0 2024-03-22T19:11:58.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.991+0000 7f13db400700 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5650122768a0 0x565012278cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:11:58.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13db400700 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5650122768a0 0x565012278cc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4b5519f0544bc2 server_cookie=d7ab3653744ddc59 in_seq=0 out_seq=0 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13c5600700 10 client.15633 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13c5600700 1 -- 172.21.15.86:0/3226807484 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565011fafcf0 con 0x5650122768a0 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13c5600700 10 client.15633 handle_command_reply: tid=0 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13f5fe4b80 1 client.15633 shutdown 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565012210020 msgr2=0x56501221e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.992+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565012210020 0x56501221e2b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f13c80966f0 tx=0x7f13c8099f80 comp rx=0 tx=0).stop 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 shutdown_connections 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5650122768a0 0x565012278cc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5650122109e0 0x56501221f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56501220ede0 0x56501221dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13c5600700 0 client.15633 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 --2- 172.21.15.86:0/3226807484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565012210020 0x56501221e2b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 >> 172.21.15.86:0/3226807484 conn(0x565012184220 msgr2=0x565012209350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 shutdown_connections 2024-03-22T19:11:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 1 -- 172.21.15.86:0/3226807484 wait complete. 2024-03-22T19:11:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.993+0000 7f13f5fe4b80 20 client.15633 trim_cache size 0 max 16384 2024-03-22T19:11:58.996 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13c00a0b10 msgr2=0x7f13c00a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13c00a0b10 0x7f13c00a2fd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f13dc19a1d0 tx=0x7f13cc00b410 comp rx=0 tx=0).stop 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 msgr2=0x7f13dc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc199290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009230 tx=0x7f13c8005250 comp rx=0 tx=0).stop 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 shutdown_connections 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f13c00a0b10 0x7f13c00a2fd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13dc100770 0x7f13dc1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13dc0ff450 0x7f13dc198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 --2- 172.21.15.86:0/2300782590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13dc0ffdb0 0x7f13dc199290 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 >> 172.21.15.86:0/2300782590 conn(0x7f13dc0fafb0 msgr2=0x7f13dc0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 shutdown_connections 2024-03-22T19:11:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:11:58.994+0000 7f13e0e00700 1 -- 172.21.15.86:0/2300782590 wait complete. 2024-03-22T19:11:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:11:59 smithi131 ceph-mon[78559]: pgmap v316: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:59.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:11:59 smithi086 ceph-mon[38032]: pgmap v316: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:11:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:11:59 smithi167 ceph-mon[47728]: pgmap v316: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:12:00.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:12:00.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 -- 172.21.15.86:0/3074396812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4280769b0 msgr2=0x7fe428076e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 --2- 172.21.15.86:0/3074396812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4280769b0 0x7fe428076e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe414009990 tx=0x7fe414009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 -- 172.21.15.86:0/3074396812 shutdown_connections 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 --2- 172.21.15.86:0/3074396812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 0x7fe428077bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 --2- 172.21.15.86:0/3074396812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe428075770 0x7fe428075b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 --2- 172.21.15.86:0/3074396812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4280769b0 0x7fe428076e30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 -- 172.21.15.86:0/3074396812 >> 172.21.15.86:0/3074396812 conn(0x7fe4280fe4c0 msgr2=0x7fe428100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 -- 172.21.15.86:0/3074396812 shutdown_connections 2024-03-22T19:12:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.242+0000 7fe42ca00700 1 -- 172.21.15.86:0/3074396812 wait complete. 2024-03-22T19:12:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.244+0000 7fe42ca00700 1 Processor -- start 2024-03-22T19:12:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.244+0000 7fe42ca00700 1 -- start start 2024-03-22T19:12:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe42ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 0x7fe42819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe42ca00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4280769b0 0x7fe42819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe42ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 0x7fe4281a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe42ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4281a4e40 con 0x7fe428077370 2024-03-22T19:12:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe42ca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4281a4f80 con 0x7fe428075770 2024-03-22T19:12:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe42ca00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4281a5110 con 0x7fe4280769b0 2024-03-22T19:12:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe425e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 0x7fe42819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe425e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 0x7fe42819d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50530/0 (socket says 172.21.15.86:50530) 2024-03-22T19:12:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe425e00700 1 -- 172.21.15.86:0/1330106566 learned_addr learned my addr 172.21.15.86:0/1330106566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.245+0000 7fe425400700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4280769b0 0x7fe42819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425400700 1 -- 172.21.15.86:0/1330106566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 msgr2=0x7fe42819d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425400700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 0x7fe42819d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425400700 1 -- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 msgr2=0x7fe4281a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe426800700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 0x7fe4281a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425400700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 0x7fe4281a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425400700 1 -- 172.21.15.86:0/1330106566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140095f0 con 0x7fe4280769b0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425e00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 0x7fe42819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe426800700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 0x7fe4281a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.246+0000 7fe425400700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4280769b0 0x7fe42819d850 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098e0 tx=0x7fe414009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.247+0000 7fe41ea00700 1 -- 172.21.15.86:0/1330106566 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41401d070 con 0x7fe4280769b0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.247+0000 7fe41ea00700 1 -- 172.21.15.86:0/1330106566 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe414021410 con 0x7fe4280769b0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.247+0000 7fe41ea00700 1 -- 172.21.15.86:0/1330106566 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414017650 con 0x7fe4280769b0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.247+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4281a5390 con 0x7fe4280769b0 2024-03-22T19:12:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.247+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4281a5760 con 0x7fe4280769b0 2024-03-22T19:12:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 Processor -- start 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 -- start start 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe41ea00700 1 -- 172.21.15.86:0/1330106566 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe414021580 con 0x7fe4280769b0 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 0x55abbde1e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1e600 0x55abbde1eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1f4a0 0x55abbde1f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd76a10 con 0x55abbde1f4a0 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd743d0 con 0x55abbde1dca0 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.254+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd6fcc0 con 0x55abbde1e600 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 0x55abbde1e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 0x55abbde1e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50538/0 (socket says 172.21.15.86:50538) 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 -- 172.21.15.86:0/4018304495 learned_addr learned my addr 172.21.15.86:0/4018304495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe425e00700 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1e600 0x55abbde1eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 -- 172.21.15.86:0/4018304495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1e600 msgr2=0x55abbde1eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1e600 0x55abbde1eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 -- 172.21.15.86:0/4018304495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1f4a0 msgr2=0x55abbde1f920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:12:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1f4a0 0x55abbde1f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.255+0000 7fe426800700 1 -- 172.21.15.86:0/4018304495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbdbf1430 con 0x55abbde1dca0 2024-03-22T19:12:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.256+0000 7fe426800700 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 0x55abbde1e0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe420009400 tx=0x7fe420009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74fea9a0229fb73 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.256+0000 7fe413000700 1 -- 172.21.15.86:0/4018304495 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200153f0 con 0x55abbde1dca0 2024-03-22T19:12:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.256+0000 7fe413000700 1 -- 172.21.15.86:0/4018304495 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe420011040 con 0x55abbde1dca0 2024-03-22T19:12:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.256+0000 7fe413000700 1 -- 172.21.15.86:0/4018304495 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420014850 con 0x55abbde1dca0 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.256+0000 7fe41ea00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe40c0a0b90 0x7fe40c0a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe41ea00700 1 -- 172.21.15.86:0/1330106566 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe4140cd390 con 0x7fe4280769b0 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 -- 172.21.15.86:0/4018304495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 msgr2=0x55abbde1e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 0x55abbde1e0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe420009400 tx=0x7fe420009920 comp rx=0 tx=0).stop 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 -- 172.21.15.86:0/4018304495 shutdown_connections 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1f4a0 0x55abbde1f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1dca0 0x55abbde1e0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/4018304495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1e600 0x55abbde1eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe4404fbb80 1 -- 172.21.15.86:0/4018304495 >> 172.21.15.86:0/4018304495 conn(0x55abbdd932a0 msgr2=0x55abbde18c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.257+0000 7fe425e00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe40c0a0b90 0x7fe40c0a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.258+0000 7fe425e00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe40c0a0b90 0x7fe40c0a3050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe4180096b0 tx=0x7fe418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.258+0000 7fe4404fbb80 1 -- 172.21.15.86:0/4018304495 shutdown_connections 2024-03-22T19:12:00.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.259+0000 7fe4404fbb80 1 -- 172.21.15.86:0/4018304495 wait complete. 2024-03-22T19:12:00.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.259+0000 7fe4404fbb80 1 -- start start 2024-03-22T19:12:00.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.260+0000 7fe4404fbb80 1 Processor -- start 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.260+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.260+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.260+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde40160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde38be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43110/0 (socket says 172.21.15.86:43110) 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 -- 172.21.15.86:0/1143621222 learned_addr learned my addr 172.21.15.86:0/1143621222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd0be00 con 0x55abbde1e600 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd0baa0 con 0x55abbde1f4a0 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425400700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde39120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdbf3720 con 0x55abbde1dca0 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe4404fbb80 10 client.0 _ll_register_callbacks cb 0x55abbdb77420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe426800700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde40160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 -- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 msgr2=0x55abbde40160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde40160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 -- 172.21.15.86:0/1143621222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 msgr2=0x55abbde39120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde39120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425e00700 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4200090b0 con 0x55abbde1dca0 2024-03-22T19:12:00.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.261+0000 7fe425400700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.262+0000 7fe425e00700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde38be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe41800f730 tx=0x7fe4180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57363df7f7a0b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.262+0000 7fe407400700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.262+0000 7fe407400700 1 -- 172.21.15.86:0/1143621222 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418017070 con 0x55abbde1dca0 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.262+0000 7fe407400700 1 -- 172.21.15.86:0/1143621222 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe418004030 con 0x55abbde1dca0 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.262+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abbde10b50 con 0x55abbde1dca0 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe407400700 1 -- 172.21.15.86:0/1143621222 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4180041f0 con 0x55abbde1dca0 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 10 client.25660 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abbdb9dbe0 con 0x55abbde1dca0 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe407400700 1 -- 172.21.15.86:0/1143621222 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abbde10b50 con 0x55abbde1dca0 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe407400700 1 -- 172.21.15.86:0/1143621222 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe418011770 con 0x55abbde1dca0 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 10 client.25660 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 20 client.25660 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde44a30 0x55abbde46e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 4 client.25660 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.263+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abbdd68780 con 0x55abbde44a30 2024-03-22T19:12:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.264+0000 7fe425400700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde44a30 0x55abbde46e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.264+0000 7fe425400700 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde44a30 0x55abbde46e50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3cbd8ba890f24e server_cookie=84dde9c6f6c8bc5b in_seq=0 out_seq=0 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.264+0000 7fe407400700 10 client.25660 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.265+0000 7fe407400700 1 -- 172.21.15.86:0/1143621222 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55abbdd68780 con 0x55abbde44a30 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.265+0000 7fe407400700 10 client.25660 handle_command_reply: tid=0 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.265+0000 7fe4404fbb80 1 client.25660 shutdown 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 msgr2=0x55abbde38be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde38be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe41800f730 tx=0x7fe4180047c0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 shutdown_connections 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde44a30 0x55abbde46e50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde40160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde38be0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1143621222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe407400700 0 client.25660 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 >> 172.21.15.86:0/1143621222 conn(0x55abbdd932a0 msgr2=0x55abbde18270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 shutdown_connections 2024-03-22T19:12:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1143621222 wait complete. 2024-03-22T19:12:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.266+0000 7fe4404fbb80 20 client.25660 trim_cache size 0 max 16384 2024-03-22T19:12:00.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.283+0000 7fe4404fbb80 1 Processor -- start 2024-03-22T19:12:00.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 -- start start 2024-03-22T19:12:00.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1dca0 0x55abbde1f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1e600 0x55abbde40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1f4a0 0x55abbde380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde13000 con 0x55abbde1dca0 2024-03-22T19:12:00.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde4bb30 con 0x55abbde1e600 2024-03-22T19:12:00.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde10690 con 0x55abbde1f4a0 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe426800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1f4a0 0x55abbde380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe425400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1e600 0x55abbde40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe426800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1f4a0 0x55abbde380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43124/0 (socket says 172.21.15.86:43124) 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.284+0000 7fe426800700 1 -- 172.21.15.86:0/301422984 learned_addr learned my addr 172.21.15.86:0/301422984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425e00700 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1dca0 0x55abbde1f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425400700 1 -- 172.21.15.86:0/301422984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1f4a0 msgr2=0x55abbde380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425400700 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1f4a0 0x55abbde380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425400700 1 -- 172.21.15.86:0/301422984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1dca0 msgr2=0x55abbde1f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425400700 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1dca0 0x55abbde1f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425400700 1 -- 172.21.15.86:0/301422984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbdd35f20 con 0x55abbde1e600 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425e00700 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1dca0 0x55abbde1f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:12:00.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe425400700 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1e600 0x55abbde40120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55abbde1ec80 tx=0x7fe4140cad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb07f6b50398ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe413a00700 1 -- 172.21.15.86:0/301422984 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41409b1a0 con 0x55abbde1e600 2024-03-22T19:12:00.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe413a00700 1 -- 172.21.15.86:0/301422984 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe41409aaa0 con 0x55abbde1e600 2024-03-22T19:12:00.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.285+0000 7fe413a00700 1 -- 172.21.15.86:0/301422984 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4140cb090 con 0x55abbde1e600 2024-03-22T19:12:00.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 -- 172.21.15.86:0/301422984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1e600 msgr2=0x55abbde40120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1e600 0x55abbde40120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55abbde1ec80 tx=0x7fe4140cad50 comp rx=0 tx=0).stop 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 -- 172.21.15.86:0/301422984 shutdown_connections 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1f4a0 0x55abbde380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1dca0 0x55abbde1f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/301422984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1e600 0x55abbde40120 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 -- 172.21.15.86:0/301422984 >> 172.21.15.86:0/301422984 conn(0x55abbdd932a0 msgr2=0x55abbde259d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 -- 172.21.15.86:0/301422984 shutdown_connections 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.286+0000 7fe4404fbb80 1 -- 172.21.15.86:0/301422984 wait complete. 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.287+0000 7fe4404fbb80 1 -- start start 2024-03-22T19:12:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.287+0000 7fe4404fbb80 1 Processor -- start 2024-03-22T19:12:00.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde3fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde3e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde3e940 con 0x55abbde1e600 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde3ea80 con 0x55abbde1f4a0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde3ec40 con 0x55abbde1dca0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe4404fbb80 10 client.0 _ll_register_callbacks cb 0x7fe41800f510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe426800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde3e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe426800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde3e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50566/0 (socket says 172.21.15.86:50566) 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe426800700 1 -- 172.21.15.86:0/1467391365 learned_addr learned my addr 172.21.15.86:0/1467391365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.288+0000 7fe425e00700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde3fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe426800700 1 -- 172.21.15.86:0/1467391365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 msgr2=0x55abbde3fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe426800700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde3fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe426800700 1 -- 172.21.15.86:0/1467391365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 msgr2=0x55abbde400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe425400700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe426800700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe426800700 1 -- 172.21.15.86:0/1467391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4281a5760 con 0x55abbde1f4a0 2024-03-22T19:12:00.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe426800700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde3e260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe420018040 tx=0x7fe420005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4ef58a567ba7e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.289+0000 7fe407400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.290+0000 7fe407400700 1 -- 172.21.15.86:0/1467391365 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420006040 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.290+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe420011040 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.290+0000 7fe407400700 1 -- 172.21.15.86:0/1467391365 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe420005720 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.290+0000 7fe407400700 1 -- 172.21.15.86:0/1467391365 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe420005920 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.290+0000 7fe4404fbb80 10 client.25673 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.290+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abbdbf5d50 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.291+0000 7fe407400700 1 -- 172.21.15.86:0/1467391365 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe420011040 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.291+0000 7fe407400700 1 -- 172.21.15.86:0/1467391365 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe42001f4a0 con 0x55abbde1f4a0 2024-03-22T19:12:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe4404fbb80 10 client.25673 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe4404fbb80 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe4404fbb80 20 client.25673 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde97920 0x55abbde99d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe4404fbb80 4 client.25673 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55abbdbbed70 con 0x55abbde97920 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.292+0000 7fe425e00700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde97920 0x55abbde99d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.293+0000 7fe425e00700 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde97920 0x55abbde99d40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e536fb0764701df server_cookie=2d55d2ad859e6074 in_seq=0 out_seq=0 2024-03-22T19:12:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.293+0000 7fe407400700 10 client.25673 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe407400700 1 -- 172.21.15.86:0/1467391365 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55abbdbbed70 con 0x55abbde97920 2024-03-22T19:12:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe407400700 10 client.25673 handle_command_reply: tid=0 2024-03-22T19:12:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 client.25673 shutdown 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 msgr2=0x55abbde3e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde3e260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe420018040 tx=0x7fe420005270 comp rx=0 tx=0).stop 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 shutdown_connections 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55abbde97920 0x55abbde99d40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe407400700 0 client.25673 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abbde1f4a0 0x55abbde3e260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abbde1dca0 0x55abbde3fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 --2- 172.21.15.86:0/1467391365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abbde1e600 0x55abbde400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.294+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 >> 172.21.15.86:0/1467391365 conn(0x55abbdd932a0 msgr2=0x55abbde255c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.295+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 shutdown_connections 2024-03-22T19:12:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.295+0000 7fe4404fbb80 1 -- 172.21.15.86:0/1467391365 wait complete. 2024-03-22T19:12:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.295+0000 7fe4404fbb80 20 client.25673 trim_cache size 0 max 16384 2024-03-22T19:12:00.297 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:12:00.297 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:12:00.297 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "18f438f6-6485-464b-acc0-6693e05b0561", 2024-03-22T19:12:00.298 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:12:00.298 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:12:00.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe40c0a0b90 msgr2=0x7fe40c0a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe40c0a0b90 0x7fe40c0a3050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe4180096b0 tx=0x7fe418009260 comp rx=0 tx=0).stop 2024-03-22T19:12:00.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4280769b0 msgr2=0x7fe42819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4280769b0 0x7fe42819d850 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098e0 tx=0x7fe414009580 comp rx=0 tx=0).stop 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 shutdown_connections 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe40c0a0b90 0x7fe40c0a3050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428077370 0x7fe4281a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe428075770 0x7fe42819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 --2- 172.21.15.86:0/1330106566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4280769b0 0x7fe42819d850 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098e0 tx=0x7fe414009580 comp rx=0 tx=0).stop 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.296+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 >> 172.21.15.86:0/1330106566 conn(0x7fe4280fe4c0 msgr2=0x7fe4280fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.297+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 shutdown_connections 2024-03-22T19:12:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.297+0000 7fe42ca00700 1 -- 172.21.15.86:0/1330106566 wait complete. 2024-03-22T19:12:00.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:12:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 -- 172.21.15.86:0/3168414449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 msgr2=0x7febc0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 --2- 172.21.15.86:0/3168414449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc0102390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7febac009990 tx=0x7febac009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 -- 172.21.15.86:0/3168414449 shutdown_connections 2024-03-22T19:12:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 --2- 172.21.15.86:0/3168414449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc01028d0 0x7febc0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 --2- 172.21.15.86:0/3168414449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc0100d40 0x7febc0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 --2- 172.21.15.86:0/3168414449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc0102390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 -- 172.21.15.86:0/3168414449 >> 172.21.15.86:0/3168414449 conn(0x7febc00fc2b0 msgr2=0x7febc00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 -- 172.21.15.86:0/3168414449 shutdown_connections 2024-03-22T19:12:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.532+0000 7febc6600700 1 -- 172.21.15.86:0/3168414449 wait complete. 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.533+0000 7febc6600700 1 Processor -- start 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 -- start start 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc0100d40 0x7febc0196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc01028d0 0x7febc019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019e8e0 con 0x7febc01028d0 2024-03-22T19:12:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019ea20 con 0x7febc0100d40 2024-03-22T19:12:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.534+0000 7febc6600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019ebb0 con 0x7febc0101f30 2024-03-22T19:12:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc0100d40 0x7febc0196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc0100d40 0x7febc0196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50580/0 (socket says 172.21.15.86:50580) 2024-03-22T19:12:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbf400700 1 -- 172.21.15.86:0/1802729587 learned_addr learned my addr 172.21.15.86:0/1802729587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbea00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbfe00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc01028d0 0x7febc019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbea00700 1 -- 172.21.15.86:0/1802729587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc0100d40 msgr2=0x7febc0196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbea00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc0100d40 0x7febc0196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbea00700 1 -- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc01028d0 msgr2=0x7febc019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbea00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc01028d0 0x7febc019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbea00700 1 -- 172.21.15.86:0/1802729587 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0095f0 con 0x7febc0101f30 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.535+0000 7febbfe00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc01028d0 0x7febc019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.536+0000 7febbea00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc01972f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febac000c00 tx=0x7febac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.536+0000 7febb7e00700 1 -- 172.21.15.86:0/1802729587 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac01d070 con 0x7febc0101f30 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.536+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc019ee30 con 0x7febc0101f30 2024-03-22T19:12:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.536+0000 7febb7e00700 1 -- 172.21.15.86:0/1802729587 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febac021410 con 0x7febc0101f30 2024-03-22T19:12:00.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.536+0000 7febb7e00700 1 -- 172.21.15.86:0/1802729587 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac021ea0 con 0x7febc0101f30 2024-03-22T19:12:00.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.536+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc019f1f0 con 0x7febc0101f30 2024-03-22T19:12:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 Processor -- start 2024-03-22T19:12:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febb7e00700 1 -- 172.21.15.86:0/1802729587 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7febac021580 con 0x7febc0101f30 2024-03-22T19:12:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 -- start start 2024-03-22T19:12:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c51ba0 0x55fd90c51fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c5bc20 0x55fd90c5c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90baa990 con 0x55fd90c5bc20 2024-03-22T19:12:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90ba8350 con 0x55fd90c52500 2024-03-22T19:12:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.541+0000 7febda10cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90ba3c40 con 0x55fd90c51ba0 2024-03-22T19:12:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.542+0000 7febbfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c51ba0 0x55fd90c51fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.542+0000 7febbfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c51ba0 0x55fd90c51fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43148/0 (socket says 172.21.15.86:43148) 2024-03-22T19:12:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.542+0000 7febbfe00700 1 -- 172.21.15.86:0/911698203 learned_addr learned my addr 172.21.15.86:0/911698203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.542+0000 7febbf400700 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.542+0000 7febbea00700 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c5bc20 0x55fd90c5c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbf400700 1 -- 172.21.15.86:0/911698203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c51ba0 msgr2=0x55fd90c51fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbf400700 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c51ba0 0x55fd90c51fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbf400700 1 -- 172.21.15.86:0/911698203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c5bc20 msgr2=0x55fd90c5c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbf400700 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c5bc20 0x55fd90c5c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbf400700 1 -- 172.21.15.86:0/911698203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd90a253b0 con 0x55fd90c52500 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbea00700 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c5bc20 0x55fd90c5c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febbf400700 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c5b690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ab0 tx=0x7febb0009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3342d9d206ca2495 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.543+0000 7febb4c00700 1 -- 172.21.15.86:0/911698203 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000e8d0 con 0x55fd90c52500 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febb7e00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7feba40a0b10 0x7feba40a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febb4c00700 1 -- 172.21.15.86:0/911698203 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febb0013470 con 0x55fd90c52500 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febb4c00700 1 -- 172.21.15.86:0/911698203 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0012630 con 0x55fd90c52500 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febb7e00700 1 -- 172.21.15.86:0/1802729587 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7febac0cd260 con 0x7febc0101f30 2024-03-22T19:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febbfe00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7feba40a0b10 0x7feba40a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febda10cb80 1 -- 172.21.15.86:0/911698203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 msgr2=0x55fd90c5b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.544+0000 7febda10cb80 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c5b690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ab0 tx=0x7febb0009dc0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.545+0000 7febbfe00700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7feba40a0b10 0x7feba40a2fd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55fd90c52a10 tx=0x7febb8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.545+0000 7febda10cb80 1 -- 172.21.15.86:0/911698203 shutdown_connections 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.545+0000 7febda10cb80 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c5bc20 0x55fd90c5c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.545+0000 7febda10cb80 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c51ba0 0x55fd90c51fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.545+0000 7febda10cb80 1 --2- 172.21.15.86:0/911698203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c5b690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.545+0000 7febda10cb80 1 -- 172.21.15.86:0/911698203 >> 172.21.15.86:0/911698203 conn(0x55fd90bc7220 msgr2=0x55fd90c4c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.546+0000 7febda10cb80 1 -- 172.21.15.86:0/911698203 shutdown_connections 2024-03-22T19:12:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.546+0000 7febda10cb80 1 -- 172.21.15.86:0/911698203 wait complete. 2024-03-22T19:12:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.546+0000 7febda10cb80 1 -- start start 2024-03-22T19:12:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.547+0000 7febda10cb80 1 Processor -- start 2024-03-22T19:12:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c52500 0x55fd90c64900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c5bc20 0x55fd90c6b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90b3fd80 con 0x55fd90c51ba0 2024-03-22T19:12:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90b3fa20 con 0x55fd90c5bc20 2024-03-22T19:12:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90a276a0 con 0x55fd90c52500 2024-03-22T19:12:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febda10cb80 10 client.0 _ll_register_callbacks cb 0x55fd909ab3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.548+0000 7febbf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36216/0 (socket says 172.21.15.86:36216) 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbf400700 1 -- 172.21.15.86:0/334216838 learned_addr learned my addr 172.21.15.86:0/334216838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c52500 0x55fd90c64900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbfe00700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c5bc20 0x55fd90c6b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 -- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c5bc20 msgr2=0x55fd90c6b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c5bc20 0x55fd90c6b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 -- 172.21.15.86:0/334216838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 msgr2=0x55fd90c643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 -- 172.21.15.86:0/334216838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb00096e0 con 0x55fd90c52500 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbfe00700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c5bc20 0x55fd90c6b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbf400700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7febbea00700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c52500 0x55fd90c64900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7febac09a900 tx=0x7febac021b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5f18c4732a039e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.549+0000 7feba9600700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7feba9600700 1 -- 172.21.15.86:0/334216838 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac09a060 con 0x55fd90c52500 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7feba9600700 1 -- 172.21.15.86:0/334216838 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7febac09b070 con 0x55fd90c52500 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd90c44ad0 con 0x55fd90c52500 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7feba9600700 1 -- 172.21.15.86:0/334216838 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7febac09a970 con 0x55fd90c52500 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7febda10cb80 10 client.25678 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd909d1b60 con 0x55fd90c52500 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7feba9600700 1 -- 172.21.15.86:0/334216838 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd90c44ad0 con 0x55fd90c52500 2024-03-22T19:12:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.550+0000 7feba9600700 1 -- 172.21.15.86:0/334216838 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febac0a2960 con 0x55fd90c52500 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febda10cb80 10 client.25678 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febda10cb80 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febda10cb80 20 client.25678 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febda10cb80 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90c6fbe0 0x55fd90c72000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febda10cb80 4 client.25678 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd90b9c700 con 0x55fd90c6fbe0 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.551+0000 7febbf400700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90c6fbe0 0x55fd90c72000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.552+0000 7febbf400700 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90c6fbe0 0x55fd90c72000 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14882e4d9f0f619d server_cookie=c44bbfd281118430 in_seq=0 out_seq=0 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.552+0000 7feba9600700 10 client.25678 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.552+0000 7feba9600700 1 -- 172.21.15.86:0/334216838 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fd90b9c700 con 0x55fd90c6fbe0 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7feba9600700 10 client.25678 handle_command_reply: tid=0 2024-03-22T19:12:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 client.25678 shutdown 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c52500 msgr2=0x55fd90c64900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c52500 0x55fd90c64900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7febac09a900 tx=0x7febac021b40 comp rx=0 tx=0).stop 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 shutdown_connections 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90c6fbe0 0x55fd90c72000 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c5bc20 0x55fd90c6b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 --2- 172.21.15.86:0/334216838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c52500 0x55fd90c64900 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 >> 172.21.15.86:0/334216838 conn(0x55fd90bc7220 msgr2=0x55fd90c4c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7feba9600700 0 client.25678 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 shutdown_connections 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 1 -- 172.21.15.86:0/334216838 wait complete. 2024-03-22T19:12:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.553+0000 7febda10cb80 20 client.25678 trim_cache size 0 max 16384 2024-03-22T19:12:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.571+0000 7febda10cb80 1 Processor -- start 2024-03-22T19:12:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.571+0000 7febda10cb80 1 -- start start 2024-03-22T19:12:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febda10cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febda10cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febda10cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 0x55fd90c48a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febda10cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90c17940 con 0x55fd90c51ba0 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febda10cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90c63f30 con 0x55fd90c52500 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febda10cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90c495e0 con 0x55fd90c5bc20 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c638d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36230/0 (socket says 172.21.15.86:36230) 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 -- 172.21.15.86:0/2252247738 learned_addr learned my addr 172.21.15.86:0/2252247738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbea00700 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 -- 172.21.15.86:0/2252247738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 msgr2=0x55fd90c48a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 0x55fd90c48a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 -- 172.21.15.86:0/2252247738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 msgr2=0x55fd90c6b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.572+0000 7febbf400700 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c6b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.573+0000 7febbf400700 1 -- 172.21.15.86:0/2252247738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd90b69ea0 con 0x55fd90c51ba0 2024-03-22T19:12:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.573+0000 7febbf400700 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c638d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7febb0009a80 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4257c862191464c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.573+0000 7febb5600700 1 -- 172.21.15.86:0/2252247738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0025400 con 0x55fd90c51ba0 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.573+0000 7febb5600700 1 -- 172.21.15.86:0/2252247738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febb0007e90 con 0x55fd90c51ba0 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.573+0000 7febb5600700 1 -- 172.21.15.86:0/2252247738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0025e80 con 0x55fd90c51ba0 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 -- 172.21.15.86:0/2252247738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 msgr2=0x55fd90c638d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c638d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7febb0009a80 tx=0x7febb0009040 comp rx=0 tx=0).stop 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 -- 172.21.15.86:0/2252247738 shutdown_connections 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 0x55fd90c48a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c638d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 --2- 172.21.15.86:0/2252247738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c6b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 -- 172.21.15.86:0/2252247738 >> 172.21.15.86:0/2252247738 conn(0x55fd90bc7220 msgr2=0x55fd90c621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.574+0000 7febda10cb80 1 -- 172.21.15.86:0/2252247738 shutdown_connections 2024-03-22T19:12:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.575+0000 7febda10cb80 1 -- 172.21.15.86:0/2252247738 wait complete. 2024-03-22T19:12:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.575+0000 7febda10cb80 1 -- start start 2024-03-22T19:12:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.575+0000 7febda10cb80 1 Processor -- start 2024-03-22T19:12:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 0x55fd90c6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90c6ae50 con 0x55fd90c51ba0 2024-03-22T19:12:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90c6afc0 con 0x55fd90c52500 2024-03-22T19:12:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd90c6b180 con 0x55fd90c5bc20 2024-03-22T19:12:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febda10cb80 10 client.0 _ll_register_callbacks cb 0x55fd909ab3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36234/0 (socket says 172.21.15.86:36234) 2024-03-22T19:12:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 -- 172.21.15.86:0/1597224010 learned_addr learned my addr 172.21.15.86:0/1597224010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 -- 172.21.15.86:0/1597224010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 msgr2=0x55fd90c6a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 0x55fd90c6a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 -- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 msgr2=0x55fd90c678f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c678f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.577+0000 7febbf400700 1 -- 172.21.15.86:0/1597224010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd90a253b0 con 0x55fd90c51ba0 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7febbf400700 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c673b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7febb00090b0 tx=0x7febb0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0dccd135cf382c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7feba9600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7feba9600700 1 -- 172.21.15.86:0/1597224010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0025e00 con 0x55fd90c51ba0 2024-03-22T19:12:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febb0013470 con 0x55fd90c51ba0 2024-03-22T19:12:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7feba9600700 1 -- 172.21.15.86:0/1597224010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7febb0021040 con 0x55fd90c51ba0 2024-03-22T19:12:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7feba9600700 1 -- 172.21.15.86:0/1597224010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb000ea90 con 0x55fd90c51ba0 2024-03-22T19:12:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7febda10cb80 10 client.15651 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.578+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd90a29cd0 con 0x55fd90c51ba0 2024-03-22T19:12:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.579+0000 7feba9600700 1 -- 172.21.15.86:0/1597224010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febb0013470 con 0x55fd90c51ba0 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.579+0000 7feba9600700 1 -- 172.21.15.86:0/1597224010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febb0011070 con 0x55fd90c51ba0 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febda10cb80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febda10cb80 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febda10cb80 20 client.15651 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febda10cb80 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90cc2070 0x55fd90cc4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febda10cb80 4 client.15651 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fd909f2cf0 con 0x55fd90cc2070 2024-03-22T19:12:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febbea00700 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90cc2070 0x55fd90cc4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7feba9600700 10 client.15651 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.580+0000 7febbea00700 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90cc2070 0x55fd90cc4490 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=407e8535e2be2ab7 server_cookie=907964e861bc8472 in_seq=0 out_seq=0 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7feba9600700 1 -- 172.21.15.86:0/1597224010 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fd909f2cf0 con 0x55fd90cc2070 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7feba9600700 10 client.15651 handle_command_reply: tid=0 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 client.15651 shutdown 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 msgr2=0x55fd90c673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c673b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7febb00090b0 tx=0x7febb0009fd0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 shutdown_connections 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55fd90cc2070 0x55fd90cc4490 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd90c5bc20 0x55fd90c6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd90c51ba0 0x55fd90c673b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 --2- 172.21.15.86:0/1597224010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd90c52500 0x55fd90c678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7feba9600700 0 client.15651 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 >> 172.21.15.86:0/1597224010 conn(0x55fd90bc7220 msgr2=0x55fd90c62020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 shutdown_connections 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.581+0000 7febda10cb80 1 -- 172.21.15.86:0/1597224010 wait complete. 2024-03-22T19:12:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.582+0000 7febda10cb80 20 client.15651 trim_cache size 0 max 16384 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: "18f438f6-6485-464b-acc0-6693e05b0561": { 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "18f438f6-6485-464b-acc0-6693e05b0561", 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:12:00.585 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.582+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7feba40a0b10 msgr2=0x7feba40a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.582+0000 7febc6600700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7feba40a0b10 0x7feba40a2fd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55fd90c52a10 tx=0x7febb8009040 comp rx=0 tx=0).stop 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 msgr2=0x7febc01972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc01972f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febac000c00 tx=0x7febac005740 comp rx=0 tx=0).stop 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 shutdown_connections 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7feba40a0b10 0x7feba40a2fd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc01028d0 0x7febc019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febc0100d40 0x7febc0196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 --2- 172.21.15.86:0/1802729587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc0101f30 0x7febc01972f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 >> 172.21.15.86:0/1802729587 conn(0x7febc00fc2b0 msgr2=0x7febc0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 shutdown_connections 2024-03-22T19:12:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:00.583+0000 7febc6600700 1 -- 172.21.15.86:0/1802729587 wait complete. 2024-03-22T19:12:00.612 INFO:tasks.cephfs.filesystem:scrub status for tag:18f438f6-6485-464b-acc0-6693e05b0561 - {'path': '/', 'tag': '18f438f6-6485-464b-acc0-6693e05b0561', 'options': 'recursive,force'} 2024-03-22T19:12:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:01 smithi086 ceph-mon[38032]: pgmap v317: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:01 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:12:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:01 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:01 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:12:01.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:01 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:01 smithi167 ceph-mon[47728]: pgmap v317: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:01 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:12:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:01 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:01 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:12:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:01 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:01 smithi131 ceph-mon[78559]: pgmap v317: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:01 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:12:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:01 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:01 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:12:01.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:01 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:03.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:03 smithi086 ceph-mon[38032]: pgmap v318: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-22T19:12:03.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:03 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:12:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:03 smithi167 ceph-mon[47728]: pgmap v318: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-22T19:12:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:03 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:12:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:03 smithi131 ceph-mon[78559]: pgmap v318: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-22T19:12:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:03 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:12:05.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:05 smithi086 ceph-mon[38032]: pgmap v319: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-22T19:12:05.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:05 smithi167 ceph-mon[47728]: pgmap v319: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-22T19:12:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:05 smithi131 ceph-mon[78559]: pgmap v319: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-22T19:12:07.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:07 smithi131 ceph-mon[78559]: pgmap v320: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-22T19:12:07.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:07 smithi086 ceph-mon[38032]: pgmap v320: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-22T19:12:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:07 smithi167 ceph-mon[47728]: pgmap v320: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-22T19:12:09.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:09 smithi086 ceph-mon[38032]: pgmap v321: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-22T19:12:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:09 smithi167 ceph-mon[47728]: pgmap v321: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-22T19:12:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:09 smithi131 ceph-mon[78559]: pgmap v321: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-22T19:12:11.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:11 smithi086 ceph-mon[38032]: pgmap v322: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-22T19:12:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:11 smithi167 ceph-mon[47728]: pgmap v322: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-22T19:12:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:11 smithi131 ceph-mon[78559]: pgmap v322: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-22T19:12:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:13 smithi167 ceph-mon[47728]: pgmap v323: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-22T19:12:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:13 smithi131 ceph-mon[78559]: pgmap v323: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-22T19:12:13.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:13 smithi086 ceph-mon[38032]: pgmap v323: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-22T19:12:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:15 smithi167 ceph-mon[47728]: pgmap v324: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-22T19:12:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:15 smithi131 ceph-mon[78559]: pgmap v324: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-22T19:12:15.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:15 smithi086 ceph-mon[38032]: pgmap v324: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-22T19:12:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:17 smithi131 ceph-mon[78559]: pgmap v325: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:12:17.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:17 smithi086 ceph-mon[38032]: pgmap v325: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:12:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:17 smithi167 ceph-mon[47728]: pgmap v325: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-22T19:12:18.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:18 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:12:18.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:18 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:12:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:18 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:12:19.301 INFO:tasks.workunit.client.0.smithi086.stderr:Updating files: 62% (7620/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-22T19:12:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:19 smithi131 ceph-mon[78559]: pgmap v326: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:12:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:12:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:12:19.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:12:19.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:19 smithi086 ceph-mon[38032]: pgmap v326: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:12:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:12:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:12:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:12:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:19 smithi167 ceph-mon[47728]: pgmap v326: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-22T19:12:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:12:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:12:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:12:20.134 INFO:tasks.workunit.client.0.smithi086.stderr:Note: switching to '899aeb57f4ac58b35d5b52546466a4c7cb3bdc46'. 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: git switch -c 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr:Or undo this operation with: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.135 INFO:tasks.workunit.client.0.smithi086.stderr: git switch - 2024-03-22T19:12:20.136 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.136 INFO:tasks.workunit.client.0.smithi086.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T19:12:20.136 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-22T19:12:20.136 INFO:tasks.workunit.client.0.smithi086.stderr:HEAD is now at 899aeb57f4a cephadm: create ceph-exporter sock dir if it's not present 2024-03-22T19:12:20.148 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-22T19:12:20.200 INFO:tasks.workunit.client.0.smithi086.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T19:12:20.204 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T19:12:20.204 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T19:12:20.788 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T19:12:20.886 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T19:12:20.949 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T19:12:20.953 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T19:12:20.953 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T19:12:21.016 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T19:12:21.023 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:12:21.023 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-22T19:12:21.064 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-03-22T19:12:21.066 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-03-22T19:12:21.066 DEBUG:teuthology.orchestra.run.smithi086:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-03-22T19:12:21.171 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into 'ffsb'... 2024-03-22T19:12:21.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:21 smithi086 ceph-mon[38032]: pgmap v327: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:21 smithi167 ceph-mon[47728]: pgmap v327: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:21 smithi131 ceph-mon[78559]: pgmap v327: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:22.479 INFO:tasks.workunit.client.0.smithi086.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-03-22T19:12:23.493 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether build environment is sane... yes 2024-03-22T19:12:23.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:23 smithi086 ceph-mon[38032]: pgmap v328: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:23.513 INFO:tasks.workunit.client.0.smithi086.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-03-22T19:12:23.513 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gawk... gawk 2024-03-22T19:12:23.544 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether make sets $(MAKE)... yes 2024-03-22T19:12:23.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:23 smithi167 ceph-mon[47728]: pgmap v328: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-03-22T19:12:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-03-22T19:12:23.636 INFO:tasks.workunit.client.0.smithi086.stdout:checking for style of include used by make... GNU 2024-03-22T19:12:23.641 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc... gcc 2024-03-22T19:12:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:23 smithi131 ceph-mon[78559]: pgmap v328: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-22T19:12:23.737 INFO:tasks.workunit.client.0.smithi086.stdout:checking for C compiler default output file name... a.out 2024-03-22T19:12:23.742 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether the C compiler works... yes 2024-03-22T19:12:23.746 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether we are cross compiling... no 2024-03-22T19:12:23.806 INFO:tasks.workunit.client.0.smithi086.stdout:checking for suffix of executables... 2024-03-22T19:12:23.864 INFO:tasks.workunit.client.0.smithi086.stdout:checking for suffix of object files... o 2024-03-22T19:12:23.919 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether we are using the GNU C compiler... yes 2024-03-22T19:12:23.977 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether gcc accepts -g... yes 2024-03-22T19:12:24.052 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-22T19:12:24.191 INFO:tasks.workunit.client.0.smithi086.stdout:checking dependency style of gcc... gcc3 2024-03-22T19:12:24.284 INFO:tasks.workunit.client.0.smithi086.stdout:checking how to run the C preprocessor... gcc -E 2024-03-22T19:12:24.357 INFO:tasks.workunit.client.0.smithi086.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-03-22T19:12:24.360 INFO:tasks.workunit.client.0.smithi086.stdout:checking for egrep... /usr/bin/grep -E 2024-03-22T19:12:24.378 INFO:tasks.workunit.client.0.smithi086.stdout:checking for AIX... no 2024-03-22T19:12:24.403 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc... (cached) gcc 2024-03-22T19:12:24.423 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-22T19:12:24.425 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-22T19:12:24.425 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-22T19:12:24.425 INFO:tasks.workunit.client.0.smithi086.stdout:checking dependency style of gcc... (cached) gcc3 2024-03-22T19:12:24.524 INFO:tasks.workunit.client.0.smithi086.stdout:checking for main in -lm... yes 2024-03-22T19:12:24.608 INFO:tasks.workunit.client.0.smithi086.stdout:checking for main in -lpthread... yes 2024-03-22T19:12:24.862 INFO:tasks.workunit.client.0.smithi086.stdout:checking for ANSI C header files... yes 2024-03-22T19:12:24.936 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-03-22T19:12:25.011 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/types.h... yes 2024-03-22T19:12:25.092 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/stat.h... yes 2024-03-22T19:12:25.173 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stdlib.h... yes 2024-03-22T19:12:25.256 INFO:tasks.workunit.client.0.smithi086.stdout:checking for string.h... yes 2024-03-22T19:12:25.342 INFO:tasks.workunit.client.0.smithi086.stdout:checking for memory.h... yes 2024-03-22T19:12:25.424 INFO:tasks.workunit.client.0.smithi086.stdout:checking for strings.h... yes 2024-03-22T19:12:25.506 INFO:tasks.workunit.client.0.smithi086.stdout:checking for inttypes.h... yes 2024-03-22T19:12:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:25 smithi167 ceph-mon[47728]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 106 KiB/s wr, 16 op/s 2024-03-22T19:12:25.603 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stdint.h... yes 2024-03-22T19:12:25.693 INFO:tasks.workunit.client.0.smithi086.stdout:checking for unistd.h... yes 2024-03-22T19:12:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:25 smithi131 ceph-mon[78559]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 106 KiB/s wr, 16 op/s 2024-03-22T19:12:25.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:25 smithi086 ceph-mon[38032]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 106 KiB/s wr, 16 op/s 2024-03-22T19:12:25.789 INFO:tasks.workunit.client.0.smithi086.stdout:checking pthread.h usability... yes 2024-03-22T19:12:25.825 INFO:tasks.workunit.client.0.smithi086.stdout:checking pthread.h presence... yes 2024-03-22T19:12:25.826 INFO:tasks.workunit.client.0.smithi086.stdout:checking for pthread.h... yes 2024-03-22T19:12:25.917 INFO:tasks.workunit.client.0.smithi086.stdout:checking fcntl.h usability... yes 2024-03-22T19:12:25.953 INFO:tasks.workunit.client.0.smithi086.stdout:checking fcntl.h presence... yes 2024-03-22T19:12:25.954 INFO:tasks.workunit.client.0.smithi086.stdout:checking for fcntl.h... yes 2024-03-22T19:12:26.046 INFO:tasks.workunit.client.0.smithi086.stdout:checking limits.h usability... yes 2024-03-22T19:12:26.083 INFO:tasks.workunit.client.0.smithi086.stdout:checking limits.h presence... yes 2024-03-22T19:12:26.084 INFO:tasks.workunit.client.0.smithi086.stdout:checking for limits.h... yes 2024-03-22T19:12:26.107 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stdint.h... (cached) yes 2024-03-22T19:12:26.201 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/time.h usability... yes 2024-03-22T19:12:26.238 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/time.h presence... yes 2024-03-22T19:12:26.239 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/time.h... yes 2024-03-22T19:12:26.264 INFO:tasks.workunit.client.0.smithi086.stdout:checking for unistd.h... (cached) yes 2024-03-22T19:12:26.358 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/vfs.h usability... yes 2024-03-22T19:12:26.393 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/vfs.h presence... yes 2024-03-22T19:12:26.394 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/vfs.h... yes 2024-03-22T19:12:26.471 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/limits.h usability... no 2024-03-22T19:12:26.506 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/limits.h presence... no 2024-03-22T19:12:26.507 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/limits.h... no 2024-03-22T19:12:26.568 INFO:tasks.workunit.client.0.smithi086.stdout:checking for an ANSI C-conforming const... yes 2024-03-22T19:12:26.696 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-03-22T19:12:26.698 INFO:tasks.workunit.client.0.smithi086.stdout:checking for function prototypes... yes 2024-03-22T19:12:26.756 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether setvbuf arguments are reversed... no 2024-03-22T19:12:26.896 INFO:tasks.workunit.client.0.smithi086.stdout:checking for system... yes 2024-03-22T19:12:27.002 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gettimeofday... yes 2024-03-22T19:12:27.108 INFO:tasks.workunit.client.0.smithi086.stdout:checking for mkdir... yes 2024-03-22T19:12:27.215 INFO:tasks.workunit.client.0.smithi086.stdout:checking for strerror... yes 2024-03-22T19:12:27.322 INFO:tasks.workunit.client.0.smithi086.stdout:checking for open64... yes 2024-03-22T19:12:27.442 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stat64... yes 2024-03-22T19:12:27.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:27 smithi131 ceph-mon[78559]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 107 KiB/s wr, 16 op/s 2024-03-22T19:12:27.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:27 smithi086 ceph-mon[38032]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 107 KiB/s wr, 16 op/s 2024-03-22T19:12:27.549 INFO:tasks.workunit.client.0.smithi086.stdout:checking for fseeko64... yes 2024-03-22T19:12:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:27 smithi167 ceph-mon[47728]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 107 KiB/s wr, 16 op/s 2024-03-22T19:12:27.658 INFO:tasks.workunit.client.0.smithi086.stdout:checking for lrand48_r... yes 2024-03-22T19:12:27.770 INFO:tasks.workunit.client.0.smithi086.stdout:checking for srand48_r... yes 2024-03-22T19:12:27.816 INFO:tasks.workunit.client.0.smithi086.stdout:configure: creating ./config.status 2024-03-22T19:12:28.025 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: creating Makefile 2024-03-22T19:12:28.057 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: creating config.h 2024-03-22T19:12:28.090 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: executing depfiles commands 2024-03-22T19:12:28.188 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: executing default commands 2024-03-22T19:12:28.264 INFO:tasks.workunit.client.0.smithi086.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-03-22T19:12:28.269 INFO:tasks.workunit.client.0.smithi086.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-03-22T19:12:28.271 INFO:tasks.workunit.client.0.smithi086.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-03-22T19:12:28.271 INFO:tasks.workunit.client.0.smithi086.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-03-22T19:12:28.271 INFO:tasks.workunit.client.0.smithi086.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-03-22T19:12:28.271 INFO:tasks.workunit.client.0.smithi086.stderr: any GNU archive site. 2024-03-22T19:12:28.275 INFO:tasks.workunit.client.0.smithi086.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-03-22T19:12:28.278 INFO:tasks.workunit.client.0.smithi086.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-03-22T19:12:28.280 INFO:tasks.workunit.client.0.smithi086.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-03-22T19:12:28.280 INFO:tasks.workunit.client.0.smithi086.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-03-22T19:12:28.280 INFO:tasks.workunit.client.0.smithi086.stderr: You might want to install the `Automake' and `Perl' packages. 2024-03-22T19:12:28.280 INFO:tasks.workunit.client.0.smithi086.stderr: Grab them from any GNU archive site. 2024-03-22T19:12:28.338 INFO:tasks.workunit.client.0.smithi086.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-03-22T19:12:28.659 INFO:tasks.workunit.client.0.smithi086.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-03-22T19:12:28.660 INFO:tasks.workunit.client.0.smithi086.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-03-22T19:12:28.660 INFO:tasks.workunit.client.0.smithi086.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-03-22T19:12:28.660 INFO:tasks.workunit.client.0.smithi086.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-03-22T19:12:28.790 INFO:tasks.workunit.client.0.smithi086.stdout:/bin/sh ./config.status --recheck 2024-03-22T19:12:28.910 INFO:tasks.workunit.client.0.smithi086.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-03-22T19:12:29.102 INFO:tasks.workunit.client.0.smithi086.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-03-22T19:12:29.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:29 smithi086 ceph-mon[38032]: pgmap v331: 97 pgs: 97 active+clean; 3.8 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 284 KiB/s wr, 27 op/s 2024-03-22T19:12:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:29 smithi167 ceph-mon[47728]: pgmap v331: 97 pgs: 97 active+clean; 3.8 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 284 KiB/s wr, 27 op/s 2024-03-22T19:12:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:29 smithi131 ceph-mon[78559]: pgmap v331: 97 pgs: 97 active+clean; 3.8 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 284 KiB/s wr, 27 op/s 2024-03-22T19:12:30.115 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether build environment is sane... yes 2024-03-22T19:12:30.128 INFO:tasks.workunit.client.0.smithi086.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-03-22T19:12:30.128 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gawk... gawk 2024-03-22T19:12:30.153 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether make sets $(MAKE)... yes 2024-03-22T19:12:30.227 INFO:tasks.workunit.client.0.smithi086.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-03-22T19:12:30.227 INFO:tasks.workunit.client.0.smithi086.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-03-22T19:12:30.244 INFO:tasks.workunit.client.0.smithi086.stdout:checking for style of include used by make... GNU 2024-03-22T19:12:30.249 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc... gcc 2024-03-22T19:12:30.385 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether the C compiler works... yes 2024-03-22T19:12:30.385 INFO:tasks.workunit.client.0.smithi086.stdout:checking for C compiler default output file name... a.out 2024-03-22T19:12:30.446 INFO:tasks.workunit.client.0.smithi086.stdout:checking for suffix of executables... 2024-03-22T19:12:30.508 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether we are cross compiling... no 2024-03-22T19:12:30.567 INFO:tasks.workunit.client.0.smithi086.stdout:checking for suffix of object files... o 2024-03-22T19:12:30.610 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether we are using the GNU C compiler... yes 2024-03-22T19:12:30.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:12:30.652 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether gcc accepts -g... yes 2024-03-22T19:12:30.709 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-22T19:12:30.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 -- 172.21.15.86:0/52762392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38100d40 msgr2=0x7f8b38101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 --2- 172.21.15.86:0/52762392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38100d40 0x7f8b38101160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b280099c0 tx=0x7f8b28009cd0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 -- 172.21.15.86:0/52762392 shutdown_connections 2024-03-22T19:12:30.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 --2- 172.21.15.86:0/52762392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b381028d0 0x7f8b38109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 --2- 172.21.15.86:0/52762392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38100d40 0x7f8b38101160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 --2- 172.21.15.86:0/52762392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b38101f30 0x7f8b38102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 -- 172.21.15.86:0/52762392 >> 172.21.15.86:0/52762392 conn(0x7f8b380fc2b0 msgr2=0x7f8b380fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:30.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 -- 172.21.15.86:0/52762392 shutdown_connections 2024-03-22T19:12:30.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.817+0000 7f8b3e000700 1 -- 172.21.15.86:0/52762392 wait complete. 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 Processor -- start 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 -- start start 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38100d40 0x7f8b38196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 0x7f8b381972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b381028d0 0x7f8b3819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819e840 con 0x7f8b38100d40 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819e9b0 con 0x7f8b38101f30 2024-03-22T19:12:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b3e000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819eb70 con 0x7f8b381028d0 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38100d40 0x7f8b38196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b36a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 0x7f8b381972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b36a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 0x7f8b381972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34430/0 (socket says 172.21.15.86:34430) 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.818+0000 7f8b36a00700 1 -- 172.21.15.86:0/824787179 learned_addr learned my addr 172.21.15.86:0/824787179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b37e00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b381028d0 0x7f8b3819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b36a00700 1 -- 172.21.15.86:0/824787179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b381028d0 msgr2=0x7f8b3819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b36a00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b381028d0 0x7f8b3819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b36a00700 1 -- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38100d40 msgr2=0x7f8b38196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b36a00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38100d40 0x7f8b38196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b36a00700 1 -- 172.21.15.86:0/824787179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280095f0 con 0x7f8b38101f30 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b37400700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38100d40 0x7f8b38196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b36a00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 0x7f8b381972e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400e9a0 tx=0x7f8b2400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b2fe00700 1 -- 172.21.15.86:0/824787179 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24009790 con 0x7f8b38101f30 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b2fe00700 1 -- 172.21.15.86:0/824787179 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b24010430 con 0x7f8b38101f30 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3819ee50 con 0x7f8b38101f30 2024-03-22T19:12:30.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.819+0000 7f8b2fe00700 1 -- 172.21.15.86:0/824787179 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24018680 con 0x7f8b38101f30 2024-03-22T19:12:30.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.820+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3819f280 con 0x7f8b38101f30 2024-03-22T19:12:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.822+0000 7f8b2fe00700 1 -- 172.21.15.86:0/824787179 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8b240105a0 con 0x7f8b38101f30 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b51b4db80 1 Processor -- start 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b2fe00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b1c0a0b10 0x7f8b1c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b2fe00700 1 -- 172.21.15.86:0/824787179 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b24014070 con 0x7f8b38101f30 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b51b4db80 1 -- start start 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b37e00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b1c0a0b10 0x7f8b1c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1ad6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ad7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.824+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1ad7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1a2a990 con 0x556da1ad6990 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1a28350 con 0x556da1ad6030 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1a23c40 con 0x556da1ad7830 2024-03-22T19:12:30.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1ad6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ad7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1ad6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34444/0 (socket says 172.21.15.86:34444) 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 -- 172.21.15.86:0/3422357705 learned_addr learned my addr 172.21.15.86:0/3422357705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37e00700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b1c0a0b10 0x7f8b1c0a2fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000f540 tx=0x7f8b3000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 -- 172.21.15.86:0/3422357705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 msgr2=0x556da1ad7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1ad7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 -- 172.21.15.86:0/3422357705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 msgr2=0x556da1ad7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ad7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 -- 172.21.15.86:0/3422357705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556da18a53b0 con 0x556da1ad6030 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.825+0000 7f8b37400700 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1ad6450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006010 tx=0x7f8b28005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f99176ad93ee4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b23e00700 1 -- 172.21.15.86:0/3422357705 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bdc0 con 0x556da1ad6030 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b23e00700 1 -- 172.21.15.86:0/3422357705 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b2801e040 con 0x556da1ad6030 2024-03-22T19:12:30.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b23e00700 1 -- 172.21.15.86:0/3422357705 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800f460 con 0x556da1ad6030 2024-03-22T19:12:30.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3422357705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 msgr2=0x556da1ad6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1ad6450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006010 tx=0x7f8b28005c00 comp rx=0 tx=0).stop 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3422357705 shutdown_connections 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1ad7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1ad6450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3422357705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ad7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3422357705 >> 172.21.15.86:0/3422357705 conn(0x556da1a47220 msgr2=0x556da1accbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3422357705 shutdown_connections 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.826+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3422357705 wait complete. 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.827+0000 7f8b51b4db80 1 -- start start 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.827+0000 7f8b51b4db80 1 Processor -- start 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.827+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 0x556da1adff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.827+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6990 0x556da1ae04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.827+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ae7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.827+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da19bfd80 con 0x556da1ad6030 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da19bfa20 con 0x556da1ad7830 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da18a76a0 con 0x556da1ad6990 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b51b4db80 10 client.0 _ll_register_callbacks cb 0x556da182b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 0x556da1adff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 0x556da1adff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60866/0 (socket says 172.21.15.86:60866) 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b36a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6990 0x556da1ae04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 -- 172.21.15.86:0/1759964960 learned_addr learned my addr 172.21.15.86:0/1759964960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37e00700 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ae7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 -- 172.21.15.86:0/1759964960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6990 msgr2=0x556da1ae04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6990 0x556da1ae04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 -- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 msgr2=0x556da1ae7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ae7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 -- 172.21.15.86:0/1759964960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b38109ad0 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b37400700 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 0x556da1adff60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098e0 tx=0x7f8b2800f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20c70f515696dc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.828+0000 7f8b20c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b20c00700 1 -- 172.21.15.86:0/1759964960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2802f070 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556da1ac4ad0 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b20c00700 1 -- 172.21.15.86:0/1759964960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b28005280 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b20c00700 1 -- 172.21.15.86:0/1759964960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b28005940 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b51b4db80 10 client.15663 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556da1851b60 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b20c00700 1 -- 172.21.15.86:0/1759964960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556da1ac4ad0 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b20c00700 1 -- 172.21.15.86:0/1759964960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b28033020 con 0x556da1ad6030 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b51b4db80 10 client.15663 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.829+0000 7f8b51b4db80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b51b4db80 20 client.15663 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1aeb6e0 0x556da1aedb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b51b4db80 4 client.15663 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:30.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556da1a1c700 con 0x556da1aeb6e0 2024-03-22T19:12:30.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b36a00700 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1aeb6e0 0x556da1aedb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b20c00700 10 client.15663 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:30.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.830+0000 7f8b36a00700 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1aeb6e0 0x556da1aedb00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8830a11840b3bb9 server_cookie=3d005d30d1ced8e in_seq=0 out_seq=0 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.832+0000 7f8b20c00700 1 -- 172.21.15.86:0/1759964960 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556da1a1c700 con 0x556da1aeb6e0 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.832+0000 7f8b20c00700 10 client.15663 handle_command_reply: tid=0 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 client.15663 shutdown 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 msgr2=0x556da1adff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 0x556da1adff60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098e0 tx=0x7f8b2800f4e0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 shutdown_connections 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1aeb6e0 0x556da1aedb00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ae7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6030 0x556da1adff60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/1759964960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6990 0x556da1ae04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b20c00700 0 client.15663 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:30.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 >> 172.21.15.86:0/1759964960 conn(0x556da1a47220 msgr2=0x556da1acc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:30.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 shutdown_connections 2024-03-22T19:12:30.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 1 -- 172.21.15.86:0/1759964960 wait complete. 2024-03-22T19:12:30.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.833+0000 7f8b51b4db80 20 client.15663 trim_cache size 0 max 16384 2024-03-22T19:12:30.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 Processor -- start 2024-03-22T19:12:30.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 -- start start 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6030 0x556da1acd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ae53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ad7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1ade650 con 0x556da1ad6990 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1ac95e0 con 0x556da1ad7830 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.850+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1ac4640 con 0x556da1ad6030 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ae53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b37e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ad7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b37e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ad7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34474/0 (socket says 172.21.15.86:34474) 2024-03-22T19:12:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b37400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6030 0x556da1acd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b37e00700 1 -- 172.21.15.86:0/207695054 learned_addr learned my addr 172.21.15.86:0/207695054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 -- 172.21.15.86:0/207695054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6030 msgr2=0x556da1acd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6030 0x556da1acd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 -- 172.21.15.86:0/207695054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 msgr2=0x556da1ad7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ad7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 -- 172.21.15.86:0/207695054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556da19e9ea0 con 0x556da1ad6990 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b37e00700 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ad7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b36a00700 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ae53f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c2d0 tx=0x7f8b24010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6571a3547ae81c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.851+0000 7f8b2d200700 1 -- 172.21.15.86:0/207695054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240a1c80 con 0x556da1ad6990 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b2d200700 1 -- 172.21.15.86:0/207695054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b240c9d80 con 0x556da1ad6990 2024-03-22T19:12:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b2d200700 1 -- 172.21.15.86:0/207695054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24099a60 con 0x556da1ad6990 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 -- 172.21.15.86:0/207695054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 msgr2=0x556da1ae53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ae53f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c2d0 tx=0x7f8b24010c80 comp rx=0 tx=0).stop 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 -- 172.21.15.86:0/207695054 shutdown_connections 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad7830 0x556da1ad7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad6030 0x556da1acd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/207695054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1ae53f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 -- 172.21.15.86:0/207695054 >> 172.21.15.86:0/207695054 conn(0x556da1a47220 msgr2=0x556da1ac8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 -- 172.21.15.86:0/207695054 shutdown_connections 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.852+0000 7f8b51b4db80 1 -- 172.21.15.86:0/207695054 wait complete. 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.853+0000 7f8b51b4db80 1 -- start start 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.853+0000 7f8b51b4db80 1 Processor -- start 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1b35410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1b35950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1b3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1b3c940 con 0x556da1ad6990 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1b3ca80 con 0x556da1ad6030 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da1b3cc10 con 0x556da1ad7830 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b51b4db80 10 client.0 _ll_register_callbacks cb 0x7f8b24005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1b35410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1b35410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34486/0 (socket says 172.21.15.86:34486) 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 -- 172.21.15.86:0/3045042087 learned_addr learned my addr 172.21.15.86:0/3045042087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b36a00700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1b35950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37e00700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1b3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 -- 172.21.15.86:0/3045042087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 msgr2=0x556da1b3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1b3c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 -- 172.21.15.86:0/3045042087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 msgr2=0x556da1b35950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1b35950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 -- 172.21.15.86:0/3045042087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b30009040 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b36a00700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1b35950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b37400700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1b35410 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b28017590 tx=0x7f8b28000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1c65c06847ac1a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.854+0000 7f8b20c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b20c00700 1 -- 172.21.15.86:0/3045042087 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2802f040 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b2801e040 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b20c00700 1 -- 172.21.15.86:0/3045042087 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8b28000e40 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b20c00700 1 -- 172.21.15.86:0/3045042087 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b28005bc0 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 10 client.25703 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8b3819f280 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b20c00700 1 -- 172.21.15.86:0/3045042087 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b28044020 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b20c00700 1 -- 172.21.15.86:0/3045042087 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b28032480 con 0x556da1ad6030 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 10 client.25703 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 20 client.25703 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1b3e1e0 0x556da1b405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 4 client.25703 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.855+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556da1872cf0 con 0x556da1b3e1e0 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.856+0000 7f8b36a00700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1b3e1e0 0x556da1b405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.856+0000 7f8b20c00700 10 client.25703 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.856+0000 7f8b36a00700 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1b3e1e0 0x556da1b405d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ebce1df8a49b5c server_cookie=7c510730e61beeed in_seq=0 out_seq=0 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b20c00700 1 -- 172.21.15.86:0/3045042087 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556da1872cf0 con 0x556da1b3e1e0 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b20c00700 10 client.25703 handle_command_reply: tid=0 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 client.25703 shutdown 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 msgr2=0x556da1b35410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1b35410 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b28017590 tx=0x7f8b28000bc0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 shutdown_connections 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556da1b3e1e0 0x556da1b405d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556da1ad7830 0x556da1b3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556da1ad6030 0x556da1b35410 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.862 INFO:tasks.workunit.client.0.smithi086.stdout:checking dependency style of gcc... gcc3 2024-03-22T19:12:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b20c00700 0 client.25703 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 --2- 172.21.15.86:0/3045042087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da1ad6990 0x556da1b35950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 >> 172.21.15.86:0/3045042087 conn(0x556da1a47220 msgr2=0x556da1adf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 shutdown_connections 2024-03-22T19:12:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 1 -- 172.21.15.86:0/3045042087 wait complete. 2024-03-22T19:12:30.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.858+0000 7f8b51b4db80 20 client.25703 trim_cache size 0 max 16384 2024-03-22T19:12:30.863 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:12:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:12:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:12:30.863 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b1c0a0b10 msgr2=0x7f8b1c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b1c0a0b10 0x7f8b1c0a2fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000f540 tx=0x7f8b3000b040 comp rx=0 tx=0).stop 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 msgr2=0x7f8b381972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 0x7f8b381972e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400e9a0 tx=0x7f8b2400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 shutdown_connections 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8b1c0a0b10 0x7f8b1c0a2fd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b381028d0 0x7f8b3819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38100d40 0x7f8b38196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 --2- 172.21.15.86:0/824787179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b38101f30 0x7f8b381972e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 >> 172.21.15.86:0/824787179 conn(0x7f8b380fc2b0 msgr2=0x7f8b38105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:30.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 shutdown_connections 2024-03-22T19:12:30.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:30.859+0000 7f8b3e000700 1 -- 172.21.15.86:0/824787179 wait complete. 2024-03-22T19:12:30.878 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:12:30.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:12:30.926 INFO:tasks.workunit.client.0.smithi086.stdout:checking how to run the C preprocessor... gcc -E 2024-03-22T19:12:31.001 INFO:tasks.workunit.client.0.smithi086.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-03-22T19:12:31.003 INFO:tasks.workunit.client.0.smithi086.stdout:checking for egrep... /usr/bin/grep -E 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 -- 172.21.15.86:0/3383039659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c100c50 msgr2=0x7f128c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 --2- 172.21.15.86:0/3383039659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c100c50 0x7f128c101070 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f12800099c0 tx=0x7f1280009cd0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 -- 172.21.15.86:0/3383039659 shutdown_connections 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 --2- 172.21.15.86:0/3383039659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 --2- 172.21.15.86:0/3383039659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c100c50 0x7f128c101070 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 --2- 172.21.15.86:0/3383039659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c101e40 0x7f128c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 -- 172.21.15.86:0/3383039659 >> 172.21.15.86:0/3383039659 conn(0x7f128c0fc1c0 msgr2=0x7f128c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 -- 172.21.15.86:0/3383039659 shutdown_connections 2024-03-22T19:12:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.122+0000 7f1293c00700 1 -- 172.21.15.86:0/3383039659 wait complete. 2024-03-22T19:12:31.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 Processor -- start 2024-03-22T19:12:31.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 -- start start 2024-03-22T19:12:31.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c100c50 0x7f128c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c101e40 0x7f128c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c073e80 con 0x7f128c101e40 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c073ff0 con 0x7f128c1027e0 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.124+0000 7f1293c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c1a0c50 con 0x7f128c100c50 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f128be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c101e40 0x7f128c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f128be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c101e40 0x7f128c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60900/0 (socket says 172.21.15.86:60900) 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f128be00700 1 -- 172.21.15.86:0/2203446429 learned_addr learned my addr 172.21.15.86:0/2203446429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34512/0 (socket says 172.21.15.86:34512) 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291200700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c100c50 0x7f128c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 -- 172.21.15.86:0/2203446429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c100c50 msgr2=0x7f128c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c100c50 0x7f128c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 -- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c101e40 msgr2=0x7f128c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c101e40 0x7f128c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 -- 172.21.15.86:0/2203446429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800095f0 con 0x7f128c1027e0 2024-03-22T19:12:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.125+0000 7f1291c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c0736e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f128400efc0 tx=0x7f128400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.126+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284018070 con 0x7f128c1027e0 2024-03-22T19:12:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.126+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128400f040 con 0x7f128c1027e0 2024-03-22T19:12:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.126+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c1a0e80 con 0x7f128c1027e0 2024-03-22T19:12:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.126+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128400cce0 con 0x7f128c1027e0 2024-03-22T19:12:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.126+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c1a1300 con 0x7f128c1027e0 2024-03-22T19:12:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.127+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c1080d0 con 0x7f128c1027e0 2024-03-22T19:12:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.129+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f1284003a00 con 0x7f128c1027e0 2024-03-22T19:12:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.130+0000 7f1289600700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0a90 0x7f12740a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.130+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f12840cbbb0 con 0x7f128c1027e0 2024-03-22T19:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.130+0000 7f1291200700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0a90 0x7f12740a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.131+0000 7f1291200700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0a90 0x7f12740a2f50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1280009990 tx=0x7f1280000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.134+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1284095600 con 0x7f128c1027e0 2024-03-22T19:12:31.201 INFO:tasks.workunit.client.0.smithi086.stdout:checking for ANSI C header files... yes 2024-03-22T19:12:31.259 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/types.h... yes 2024-03-22T19:12:31.326 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/stat.h... yes 2024-03-22T19:12:31.393 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stdlib.h... yes 2024-03-22T19:12:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.406+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f128c0679e0 con 0x7f128c1027e0 2024-03-22T19:12:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.407+0000 7f1289600700 1 -- 172.21.15.86:0/2203446429 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f128401b320 con 0x7f128c1027e0 2024-03-22T19:12:31.408 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:12:31.408 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:10:44.254849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:12:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0a90 msgr2=0x7f12740a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0a90 0x7f12740a2f50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1280009990 tx=0x7f1280000bc0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 msgr2=0x7f128c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c0736e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f128400efc0 tx=0x7f128400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 shutdown_connections 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0a90 0x7f12740a2f50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1027e0 0x7f128c0736e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c100c50 0x7f128c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 --2- 172.21.15.86:0/2203446429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128c101e40 0x7f128c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 >> 172.21.15.86:0/2203446429 conn(0x7f128c0fc1c0 msgr2=0x7f128c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 shutdown_connections 2024-03-22T19:12:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.410+0000 7f1293c00700 1 -- 172.21.15.86:0/2203446429 wait complete. 2024-03-22T19:12:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-22T19:12:31.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:12:31.464 INFO:tasks.workunit.client.0.smithi086.stdout:checking for string.h... yes 2024-03-22T19:12:31.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:31 smithi086 ceph-mon[38032]: pgmap v332: 97 pgs: 97 active+clean; 4.1 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 315 KiB/s wr, 34 op/s 2024-03-22T19:12:31.533 INFO:tasks.workunit.client.0.smithi086.stdout:checking for memory.h... yes 2024-03-22T19:12:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:31 smithi167 ceph-mon[47728]: pgmap v332: 97 pgs: 97 active+clean; 4.1 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 315 KiB/s wr, 34 op/s 2024-03-22T19:12:31.606 INFO:tasks.workunit.client.0.smithi086.stdout:checking for strings.h... yes 2024-03-22T19:12:31.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 -- 172.21.15.86:0/217306053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c074df0 msgr2=0x7f381c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 --2- 172.21.15.86:0/217306053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c074df0 0x7f381c073240 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3810009990 tx=0x7f3810009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 -- 172.21.15.86:0/217306053 shutdown_connections 2024-03-22T19:12:31.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 --2- 172.21.15.86:0/217306053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c108c00 0x7f381c10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 --2- 172.21.15.86:0/217306053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c074df0 0x7f381c073240 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 --2- 172.21.15.86:0/217306053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381c073810 0x7f381c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 -- 172.21.15.86:0/217306053 >> 172.21.15.86:0/217306053 conn(0x7f381c0fc2b0 msgr2=0x7f381c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 -- 172.21.15.86:0/217306053 shutdown_connections 2024-03-22T19:12:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.670+0000 7f3823800700 1 -- 172.21.15.86:0/217306053 wait complete. 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.671+0000 7f3823800700 1 Processor -- start 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.671+0000 7f3823800700 1 -- start start 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3823800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381c073810 0x7f381c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3823800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 0x7f381c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3823800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c108c00 0x7f381c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3823800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1a0a00 con 0x7f381c108c00 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3823800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1a0b40 con 0x7f381c074df0 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3823800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1a0cd0 con 0x7f381c073810 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3821800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c108c00 0x7f381c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f381be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 0x7f381c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3820e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381c073810 0x7f381c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f381be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 0x7f381c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34534/0 (socket says 172.21.15.86:34534) 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f381be00700 1 -- 172.21.15.86:0/2988326729 learned_addr learned my addr 172.21.15.86:0/2988326729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.672+0000 7f3821800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c108c00 0x7f381c1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60918/0 (socket says 172.21.15.86:60918) 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f381be00700 1 -- 172.21.15.86:0/2988326729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381c073810 msgr2=0x7f381c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f381be00700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381c073810 0x7f381c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f381be00700 1 -- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c108c00 msgr2=0x7f381c1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f381be00700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c108c00 0x7f381c1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f381be00700 1 -- 172.21.15.86:0/2988326729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100095f0 con 0x7f381c074df0 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f381be00700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 0x7f381c199460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f380800d980 tx=0x7f380800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f3819600700 1 -- 172.21.15.86:0/2988326729 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380800f890 con 0x7f381c074df0 2024-03-22T19:12:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f3819600700 1 -- 172.21.15.86:0/2988326729 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3808010460 con 0x7f381c074df0 2024-03-22T19:12:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c1a0fb0 con 0x7f381c074df0 2024-03-22T19:12:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f3819600700 1 -- 172.21.15.86:0/2988326729 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080195c0 con 0x7f381c074df0 2024-03-22T19:12:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.673+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c1a1430 con 0x7f381c074df0 2024-03-22T19:12:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.676+0000 7f3819600700 1 -- 172.21.15.86:0/2988326729 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f38080105d0 con 0x7f381c074df0 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.677+0000 7f3819600700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f38040a0a90 0x7f38040a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.677+0000 7f38372efb80 1 Processor -- start 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f3819600700 1 -- 172.21.15.86:0/2988326729 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f38080cbd70 con 0x7f381c074df0 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f3820e00700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f38040a0a90 0x7f38040a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 -- start start 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x55933986a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933986ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933986b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593397c2990 con 0x55933986a580 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593397c0350 con 0x55933986b420 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f38372efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593397bbc40 con 0x559339869c20 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f3820e00700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f38040a0a90 0x7f38040a2f50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f381000b2e0 tx=0x7f3810005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.678+0000 7f381be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933986ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933986b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3820e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x55933986a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933986b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34542/0 (socket says 172.21.15.86:34542) 2024-03-22T19:12:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 -- 172.21.15.86:0/3878622403 learned_addr learned my addr 172.21.15.86:0/3878622403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.680 INFO:tasks.workunit.client.0.smithi086.stdout:checking for inttypes.h... yes 2024-03-22T19:12:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 -- 172.21.15.86:0/3878622403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 msgr2=0x55933986a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x55933986a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 -- 172.21.15.86:0/3878622403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 msgr2=0x55933986ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933986ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 -- 172.21.15.86:0/3878622403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933963d3b0 con 0x55933986b420 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f381be00700 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933986ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f3821800700 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933986b8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3814011680 tx=0x7f3814009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b833741593e378 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.679+0000 7f380e000700 1 -- 172.21.15.86:0/3878622403 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381400e040 con 0x55933986b420 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f380e000700 1 -- 172.21.15.86:0/3878622403 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381400f040 con 0x55933986b420 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f380e000700 1 -- 172.21.15.86:0/3878622403 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38140048c0 con 0x55933986b420 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 -- 172.21.15.86:0/3878622403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 msgr2=0x55933986b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933986b8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3814011680 tx=0x7f3814009920 comp rx=0 tx=0).stop 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 -- 172.21.15.86:0/3878622403 shutdown_connections 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933986b8a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x55933986a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 --2- 172.21.15.86:0/3878622403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933986ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 -- 172.21.15.86:0/3878622403 >> 172.21.15.86:0/3878622403 conn(0x5593397df220 msgr2=0x559339864bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 -- 172.21.15.86:0/3878622403 shutdown_connections 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.680+0000 7f38372efb80 1 -- 172.21.15.86:0/3878622403 wait complete. 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.681+0000 7f38372efb80 1 -- start start 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.681+0000 7f38372efb80 1 Processor -- start 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 0x559339873af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986a580 0x559339874030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933987b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559339757d80 con 0x559339869c20 2024-03-22T19:12:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559339757a20 con 0x55933986a580 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933963f6a0 con 0x55933986b420 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f38372efb80 10 client.0 _ll_register_callbacks cb 0x5593395c33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 0x559339873af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f381be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986a580 0x559339874030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 0x559339873af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60940/0 (socket says 172.21.15.86:60940) 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 -- 172.21.15.86:0/1007744767 learned_addr learned my addr 172.21.15.86:0/1007744767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3821800700 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933987b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 -- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 msgr2=0x55933987b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933987b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 -- 172.21.15.86:0/1007744767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986a580 msgr2=0x559339874030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986a580 0x559339874030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.682+0000 7f3820e00700 1 -- 172.21.15.86:0/1007744767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c1a1430 con 0x559339869c20 2024-03-22T19:12:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f3820e00700 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 0x559339873af0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f38100098e0 tx=0x7f3810009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0967b464c981c54 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f3802a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f3802a00700 1 -- 172.21.15.86:0/1007744767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810019970 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f3802a00700 1 -- 172.21.15.86:0/1007744767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3810019ad0 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55933985cad0 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f3802a00700 1 -- 172.21.15.86:0/1007744767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3810019c90 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f38372efb80 10 client.15687 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3814011330 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.683+0000 7f3802a00700 1 -- 172.21.15.86:0/1007744767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55933985cad0 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f3802a00700 1 -- 172.21.15.86:0/1007744767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f381003a070 con 0x559339869c20 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f38372efb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f38372efb80 10 client.15687 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f38372efb80 20 client.15687 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f38372efb80 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55933987f8f0 0x559339881d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f38372efb80 4 client.15687 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593397b4700 con 0x55933987f8f0 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f381be00700 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55933987f8f0 0x559339881d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f3802a00700 10 client.15687 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.684+0000 7f381be00700 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55933987f8f0 0x559339881d10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=713dc746d8d6abed server_cookie=7af3f7f8e26bdf57 in_seq=0 out_seq=0 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f3802a00700 1 -- 172.21.15.86:0/1007744767 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5593397b4700 con 0x55933987f8f0 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f3802a00700 10 client.15687 handle_command_reply: tid=0 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 client.15687 shutdown 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 msgr2=0x559339873af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 0x559339873af0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f38100098e0 tx=0x7f3810009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 shutdown_connections 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55933987f8f0 0x559339881d10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933987b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559339869c20 0x559339873af0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f3802a00700 0 client.15687 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:12:31.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 --2- 172.21.15.86:0/1007744767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986a580 0x559339874030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.685+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 >> 172.21.15.86:0/1007744767 conn(0x5593397df220 msgr2=0x5593398641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.686+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 shutdown_connections 2024-03-22T19:12:31.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.686+0000 7f38372efb80 1 -- 172.21.15.86:0/1007744767 wait complete. 2024-03-22T19:12:31.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.686+0000 7f38372efb80 20 client.15687 trim_cache size 0 max 16384 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 Processor -- start 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 -- start start 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559339869c20 0x559339865930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933987b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933986b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559339861650 con 0x55933986a580 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933985c640 con 0x559339869c20 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.699+0000 7f38372efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559339886ab0 con 0x55933986b420 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933987b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933987b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60942/0 (socket says 172.21.15.86:60942) 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 -- 172.21.15.86:0/820919006 learned_addr learned my addr 172.21.15.86:0/820919006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f3820e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559339869c20 0x559339865930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f3821800700 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933986b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 -- 172.21.15.86:0/820919006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 msgr2=0x55933986b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933986b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 -- 172.21.15.86:0/820919006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559339869c20 msgr2=0x559339865930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559339869c20 0x559339865930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 -- 172.21.15.86:0/820919006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593395e9b60 con 0x55933986a580 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f3820e00700 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559339869c20 0x559339865930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f381be00700 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933987b030 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f38080990f0 tx=0x7f38080c9730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627cdc74dce90471 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f380ea00700 1 -- 172.21.15.86:0/820919006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380809d220 con 0x55933986a580 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f380ea00700 1 -- 172.21.15.86:0/820919006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38080a1870 con 0x55933986a580 2024-03-22T19:12:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.700+0000 7f380ea00700 1 -- 172.21.15.86:0/820919006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808098b20 con 0x55933986a580 2024-03-22T19:12:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 -- 172.21.15.86:0/820919006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 msgr2=0x55933987b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933987b030 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f38080990f0 tx=0x7f38080c9730 comp rx=0 tx=0).stop 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 -- 172.21.15.86:0/820919006 shutdown_connections 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55933986b420 0x55933986b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559339869c20 0x559339865930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 --2- 172.21.15.86:0/820919006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x55933987b030 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 -- 172.21.15.86:0/820919006 >> 172.21.15.86:0/820919006 conn(0x5593397df220 msgr2=0x5593398718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 -- 172.21.15.86:0/820919006 shutdown_connections 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.701+0000 7f38372efb80 1 -- 172.21.15.86:0/820919006 wait complete. 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 -- start start 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 Processor -- start 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x5593398771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x559339877730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933987a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933987ab50 con 0x55933986a580 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933987acc0 con 0x55933986b420 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933987ae80 con 0x559339869c20 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.702+0000 7f38372efb80 10 client.0 _ll_register_callbacks cb 0x7f381000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x559339877730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x559339877730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60944/0 (socket says 172.21.15.86:60944) 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 -- 172.21.15.86:0/481484133 learned_addr learned my addr 172.21.15.86:0/481484133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f3821800700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933987a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f3820e00700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x5593398771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 -- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 msgr2=0x5593398771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x5593398771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 -- 172.21.15.86:0/481484133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 msgr2=0x55933987a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933987a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 -- 172.21.15.86:0/481484133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3814011330 con 0x55933986a580 2024-03-22T19:12:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f381be00700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x559339877730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38080a1340 tx=0x7f3808010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a42962caa1a983 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f3802a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f3802a00700 1 -- 172.21.15.86:0/481484133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808014070 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f3802a00700 1 -- 172.21.15.86:0/481484133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f38080988e0 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.703+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f381400f040 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f3802a00700 1 -- 172.21.15.86:0/481484133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3808098aa0 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 10 client.15693 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f381c1a0fb0 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f3802a00700 1 -- 172.21.15.86:0/481484133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381400f040 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f3802a00700 1 -- 172.21.15.86:0/481484133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3808098350 con 0x55933986a580 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 10 client.15693 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 20 client.15693 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5593398d22e0 0x5593398d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 4 client.15693 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:12:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.704+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55933960acf0 con 0x5593398d22e0 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f3820e00700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5593398d22e0 0x5593398d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f3820e00700 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5593398d22e0 0x5593398d4700 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce8976888b67814d server_cookie=951a16fcf350809e in_seq=0 out_seq=0 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f3802a00700 10 client.15693 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f3802a00700 1 -- 172.21.15.86:0/481484133 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55933960acf0 con 0x5593398d22e0 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f3802a00700 10 client.15693 handle_command_reply: tid=0 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f38372efb80 1 client.15693 shutdown 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 msgr2=0x559339877730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.705+0000 7f38372efb80 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x559339877730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38080a1340 tx=0x7f3808010a00 comp rx=0 tx=0).stop 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 shutdown_connections 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55933986b420 0x55933987a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5593398d22e0 0x5593398d4700 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559339869c20 0x5593398771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 --2- 172.21.15.86:0/481484133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55933986a580 0x559339877730 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 >> 172.21.15.86:0/481484133 conn(0x5593397df220 msgr2=0x559339864aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f3802a00700 0 client.15693 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:12:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 shutdown_connections 2024-03-22T19:12:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 1 -- 172.21.15.86:0/481484133 wait complete. 2024-03-22T19:12:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f38372efb80 20 client.15693 trim_cache size 0 max 16384 2024-03-22T19:12:31.711 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f38040a0a90 msgr2=0x7f38040a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f3823800700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f38040a0a90 0x7f38040a2f50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f381000b2e0 tx=0x7f3810005fd0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 msgr2=0x7f381c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.706+0000 7f3823800700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 0x7f381c199460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f380800d980 tx=0x7f380800dc90 comp rx=0 tx=0).stop 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 shutdown_connections 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f38040a0a90 0x7f38040a2f50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f381c108c00 0x7f381c1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381c073810 0x7f381c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 --2- 172.21.15.86:0/2988326729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f381c074df0 0x7f381c199460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 >> 172.21.15.86:0/2988326729 conn(0x7f381c0fc2b0 msgr2=0x7f381c104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 shutdown_connections 2024-03-22T19:12:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.707+0000 7f3823800700 1 -- 172.21.15.86:0/2988326729 wait complete. 2024-03-22T19:12:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:31 smithi131 ceph-mon[78559]: pgmap v332: 97 pgs: 97 active+clean; 4.1 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 315 KiB/s wr, 34 op/s 2024-03-22T19:12:31.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:12:31.751 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stdint.h... yes 2024-03-22T19:12:31.823 INFO:tasks.workunit.client.0.smithi086.stdout:checking for unistd.h... yes 2024-03-22T19:12:31.893 INFO:tasks.workunit.client.0.smithi086.stdout:checking minix/config.h usability... no 2024-03-22T19:12:31.932 INFO:tasks.workunit.client.0.smithi086.stdout:checking minix/config.h presence... no 2024-03-22T19:12:31.932 INFO:tasks.workunit.client.0.smithi086.stdout:checking for minix/config.h... no 2024-03-22T19:12:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 -- 172.21.15.86:0/3513977484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 msgr2=0x7f5e4c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 --2- 172.21.15.86:0/3513977484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c073c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009990 tx=0x7f5e38009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 -- 172.21.15.86:0/3513977484 shutdown_connections 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 --2- 172.21.15.86:0/3513977484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c108c10 0x7f5e4c10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 --2- 172.21.15.86:0/3513977484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4c074df0 0x7f5e4c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 --2- 172.21.15.86:0/3513977484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c073c90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 -- 172.21.15.86:0/3513977484 >> 172.21.15.86:0/3513977484 conn(0x7f5e4c0fc2c0 msgr2=0x7f5e4c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 -- 172.21.15.86:0/3513977484 shutdown_connections 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.951+0000 7f5e53e00700 1 -- 172.21.15.86:0/3513977484 wait complete. 2024-03-22T19:12:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.952+0000 7f5e53e00700 1 Processor -- start 2024-03-22T19:12:31.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 -- start start 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c074df0 0x7f5e4c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4c108c10 0x7f5e4c1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1a0a70 con 0x7f5e4c073810 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1a0bb0 con 0x7f5e4c108c10 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e53e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1a0d40 con 0x7f5e4c074df0 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e51400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e51400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60962/0 (socket says 172.21.15.86:60962) 2024-03-22T19:12:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.953+0000 7f5e51400700 1 -- 172.21.15.86:0/2104516179 learned_addr learned my addr 172.21.15.86:0/2104516179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e50a00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c074df0 0x7f5e4c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4c108c10 0x7f5e4c1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51400700 1 -- 172.21.15.86:0/2104516179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c074df0 msgr2=0x7f5e4c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51400700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c074df0 0x7f5e4c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51400700 1 -- 172.21.15.86:0/2104516179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4c108c10 msgr2=0x7f5e4c1a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51400700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4c108c10 0x7f5e4c1a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51400700 1 -- 172.21.15.86:0/2104516179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380095f0 con 0x7f5e4c073810 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e50a00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c074df0 0x7f5e4c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e51400700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c198f40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9a0 tx=0x7f5e4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e4a000700 1 -- 172.21.15.86:0/2104516179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40009790 con 0x7f5e4c073810 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e4a000700 1 -- 172.21.15.86:0/2104516179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4000cbd0 con 0x7f5e4c073810 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c1a0fc0 con 0x7f5e4c073810 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e4a000700 1 -- 172.21.15.86:0/2104516179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40018680 con 0x7f5e4c073810 2024-03-22T19:12:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.954+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c1a1440 con 0x7f5e4c073810 2024-03-22T19:12:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.957+0000 7f5e4a000700 1 -- 172.21.15.86:0/2104516179 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f5e4000cd40 con 0x7f5e4c073810 2024-03-22T19:12:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.959+0000 7f5e4a000700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5e340a0a90 0x7f5e340a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.959+0000 7f5e67a1fb80 1 Processor -- start 2024-03-22T19:12:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.959+0000 7f5e4a000700 1 -- 172.21.15.86:0/2104516179 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5e40014070 con 0x7f5e4c073810 2024-03-22T19:12:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.959+0000 7f5e50a00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5e340a0a90 0x7f5e340a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.959+0000 7f5e67a1fb80 1 -- start start 2024-03-22T19:12:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aed580 0x563db8aede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aee420 0x563db8aee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8a45990 con 0x563db8aed580 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8a43350 con 0x563db8aee420 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8a3ec40 con 0x563db8aecc20 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aed580 0x563db8aede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aed580 0x563db8aede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60966/0 (socket says 172.21.15.86:60966) 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e50a00700 1 -- 172.21.15.86:0/2782745244 learned_addr learned my addr 172.21.15.86:0/2782745244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51400700 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51e00700 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aee420 0x563db8aee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e50a00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5e340a0a90 0x7f5e340a2f50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800b7b0 tx=0x7f5e38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51400700 1 -- 172.21.15.86:0/2782745244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aee420 msgr2=0x563db8aee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51400700 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aee420 0x563db8aee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51400700 1 -- 172.21.15.86:0/2782745244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aed580 msgr2=0x563db8aede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51400700 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aed580 0x563db8aede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.960+0000 7f5e51400700 1 -- 172.21.15.86:0/2782745244 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563db88c03b0 con 0x563db8aecc20 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.961+0000 7f5e51400700 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aed040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c5b0 tx=0x7f5e40099280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68fa90661053a369 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.961+0000 7f5e3e600700 1 -- 172.21.15.86:0/2782745244 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40099920 con 0x563db8aecc20 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.961+0000 7f5e3e600700 1 -- 172.21.15.86:0/2782745244 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4009d750 con 0x563db8aecc20 2024-03-22T19:12:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.961+0000 7f5e3e600700 1 -- 172.21.15.86:0/2782745244 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e400a1b70 con 0x563db8aecc20 2024-03-22T19:12:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/2782745244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 msgr2=0x563db8aed040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aed040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c5b0 tx=0x7f5e40099280 comp rx=0 tx=0).stop 2024-03-22T19:12:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/2782745244 shutdown_connections 2024-03-22T19:12:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aee420 0x563db8aee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aed040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/2782745244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aed580 0x563db8aede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/2782745244 >> 172.21.15.86:0/2782745244 conn(0x563db8a62220 msgr2=0x563db8ae7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/2782745244 shutdown_connections 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/2782745244 wait complete. 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 -- start start 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.962+0000 7f5e67a1fb80 1 Processor -- start 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aecc20 0x563db8af6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aed580 0x563db8af7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8afe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db89dad80 con 0x563db8aee420 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db89daa20 con 0x563db8aecc20 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db88c26a0 con 0x563db8aed580 2024-03-22T19:12:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e67a1fb80 10 client.0 _ll_register_callbacks cb 0x563db88463a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e51e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8afe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e51400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aecc20 0x563db8af6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aed580 0x563db8af7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e51e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8afe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60970/0 (socket says 172.21.15.86:60970) 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.963+0000 7f5e51e00700 1 -- 172.21.15.86:0/3341957851 learned_addr learned my addr 172.21.15.86:0/3341957851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e51400700 1 -- 172.21.15.86:0/3341957851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aed580 msgr2=0x563db8af7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e51400700 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aed580 0x563db8af7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e51400700 1 -- 172.21.15.86:0/3341957851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 msgr2=0x563db8afe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e51400700 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8afe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e51400700 1 -- 172.21.15.86:0/3341957851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c1a1440 con 0x563db8aecc20 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e51400700 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aecc20 0x563db8af6af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c5b0 tx=0x7f5e400955f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a293d680a584423 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e32a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e32a00700 1 -- 172.21.15.86:0/3341957851 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40099060 con 0x563db8aecc20 2024-03-22T19:12:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563db8adfad0 con 0x563db8aecc20 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e32a00700 1 -- 172.21.15.86:0/3341957851 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5e40099920 con 0x563db8aecc20 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e32a00700 1 -- 172.21.15.86:0/3341957851 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e40065260 con 0x563db8aecc20 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e67a1fb80 10 client.25739 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.964+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563db886cb60 con 0x563db8aecc20 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e32a00700 1 -- 172.21.15.86:0/3341957851 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4007e070 con 0x563db8aecc20 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e32a00700 1 -- 172.21.15.86:0/3341957851 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e400b9250 con 0x563db8aecc20 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e67a1fb80 10 client.25739 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e67a1fb80 10 client.25739 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e67a1fb80 20 client.25739 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b02340 0x563db8b04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e67a1fb80 4 client.25739 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563db8a37700 con 0x563db8b02340 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.965+0000 7f5e50a00700 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b02340 0x563db8b04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.966+0000 7f5e32a00700 10 client.25739 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.966+0000 7f5e50a00700 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b02340 0x563db8b04760 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6fb57f26aa892f75 server_cookie=7ce641266de750fd in_seq=0 out_seq=0 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.966+0000 7f5e32a00700 1 -- 172.21.15.86:0/3341957851 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563db8a37700 con 0x563db8b02340 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.966+0000 7f5e32a00700 10 client.25739 handle_command_reply: tid=0 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.966+0000 7f5e67a1fb80 1 client.25739 shutdown 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aecc20 msgr2=0x563db8af6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aecc20 0x563db8af6af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c5b0 tx=0x7f5e400955f0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 shutdown_connections 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8afe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aecc20 0x563db8af6af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b02340 0x563db8b04760 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3341957851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aed580 0x563db8af7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e32a00700 0 client.25739 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:12:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 >> 172.21.15.86:0/3341957851 conn(0x563db8a62220 msgr2=0x563db8ae71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 shutdown_connections 2024-03-22T19:12:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3341957851 wait complete. 2024-03-22T19:12:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.967+0000 7f5e67a1fb80 20 client.25739 trim_cache size 0 max 16384 2024-03-22T19:12:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 Processor -- start 2024-03-22T19:12:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 -- start start 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8af6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8ab2940 con 0x563db8aee420 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8b02150 con 0x563db8aed580 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8ae45e0 con 0x563db8aecc20 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34620/0 (socket says 172.21.15.86:34620) 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.986+0000 7f5e50a00700 1 -- 172.21.15.86:0/929851872 learned_addr learned my addr 172.21.15.86:0/929851872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e51400700 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e51e00700 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8af6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e50a00700 1 -- 172.21.15.86:0/929851872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 msgr2=0x563db8aee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e50a00700 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e50a00700 1 -- 172.21.15.86:0/929851872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 msgr2=0x563db8af6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e50a00700 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8af6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e50a00700 1 -- 172.21.15.86:0/929851872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563db8a04ea0 con 0x563db8aed580 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e51e00700 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8af6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e50a00700 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdfc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e38015000 tx=0x7f5e3801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d41c3d69c3b0768 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e3f000700 1 -- 172.21.15.86:0/929851872 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38036070 con 0x563db8aed580 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e3f000700 1 -- 172.21.15.86:0/929851872 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e38028070 con 0x563db8aed580 2024-03-22T19:12:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.987+0000 7f5e3f000700 1 -- 172.21.15.86:0/929851872 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38004a30 con 0x563db8aed580 2024-03-22T19:12:31.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/929851872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 msgr2=0x563db8afdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdfc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e38015000 tx=0x7f5e3801f000 comp rx=0 tx=0).stop 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/929851872 shutdown_connections 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aee420 0x563db8af6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aecc20 0x563db8aee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/929851872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdfc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/929851872 >> 172.21.15.86:0/929851872 conn(0x563db8a62220 msgr2=0x563db8af49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/929851872 shutdown_connections 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.988+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/929851872 wait complete. 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 -- start start 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 Processor -- start 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aecc20 0x563db8afd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aee420 0x563db8afc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8afc780 con 0x563db8aecc20 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8afc8f0 con 0x563db8aed580 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db8afcab0 con 0x563db8aee420 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e67a1fb80 10 client.0 _ll_register_callbacks cb 0x7f5e38009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e51400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aecc20 0x563db8afd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.989+0000 7f5e50a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34632/0 (socket says 172.21.15.86:34632) 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 -- 172.21.15.86:0/3249884958 learned_addr learned my addr 172.21.15.86:0/3249884958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e51e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aee420 0x563db8afc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 -- 172.21.15.86:0/3249884958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aee420 msgr2=0x563db8afc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aee420 0x563db8afc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 -- 172.21.15.86:0/3249884958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aecc20 msgr2=0x563db8afd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aecc20 0x563db8afd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 -- 172.21.15.86:0/3249884958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3802b040 con 0x563db8aed580 2024-03-22T19:12:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e50a00700 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdf20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e38005cd0 tx=0x7f5e380057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a92cd420daca78 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e32a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e32a00700 1 -- 172.21.15.86:0/3249884958 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38036070 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e32a00700 1 -- 172.21.15.86:0/3249884958 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5e38010df0 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.990+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e4009d750 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e32a00700 1 -- 172.21.15.86:0/3249884958 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e38031620 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 10 client.25745 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563db88c4cd0 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e32a00700 1 -- 172.21.15.86:0/3249884958 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4009d750 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e32a00700 1 -- 172.21.15.86:0/3249884958 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e380419c0 con 0x563db8aed580 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 10 client.25745 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 10 client.25745 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 20 client.25745 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b54e70 0x563db8b57260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 4 client.25745 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:12:31.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563db888dcf0 con 0x563db8b54e70 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.991+0000 7f5e51400700 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b54e70 0x563db8b57260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e51400700 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b54e70 0x563db8b57260 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6729c523b1f5d8d3 server_cookie=65a3b277a5a66d8b in_seq=0 out_seq=0 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e32a00700 10 client.25745 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e32a00700 1 -- 172.21.15.86:0/3249884958 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563db888dcf0 con 0x563db8b54e70 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e32a00700 10 client.25745 handle_command_reply: tid=0 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 client.25745 shutdown 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 msgr2=0x563db8afdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdf20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e38005cd0 tx=0x7f5e380057e0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 shutdown_connections 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x563db8b54e70 0x563db8b57260 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563db8aee420 0x563db8afc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563db8aecc20 0x563db8afd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e32a00700 0 client.25745 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:12:31.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 --2- 172.21.15.86:0/3249884958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563db8aed580 0x563db8afdf20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 >> 172.21.15.86:0/3249884958 conn(0x563db8a62220 msgr2=0x563db8ae7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 shutdown_connections 2024-03-22T19:12:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.992+0000 7f5e67a1fb80 1 -- 172.21.15.86:0/3249884958 wait complete. 2024-03-22T19:12:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e67a1fb80 20 client.25745 trim_cache size 0 max 16384 2024-03-22T19:12:31.998 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5e340a0a90 msgr2=0x7f5e340a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5e340a0a90 0x7f5e340a2f50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800b7b0 tx=0x7f5e38005fb0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 msgr2=0x7f5e4c198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c198f40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9a0 tx=0x7f5e4000ecb0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 shutdown_connections 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f5e340a0a90 0x7f5e340a2f50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4c108c10 0x7f5e4c1a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e4c073810 0x7f5e4c198f40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 --2- 172.21.15.86:0/2104516179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e4c074df0 0x7f5e4c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.993+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 >> 172.21.15.86:0/2104516179 conn(0x7f5e4c0fc2c0 msgr2=0x7f5e4c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.994+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 shutdown_connections 2024-03-22T19:12:32.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:31.994+0000 7f5e53e00700 1 -- 172.21.15.86:0/2104516179 wait complete. 2024-03-22T19:12:32.000 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-03-22T19:12:32.000 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc... (cached) gcc 2024-03-22T19:12:32.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:12:32.053 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-22T19:12:32.053 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-22T19:12:32.053 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-22T19:12:32.053 INFO:tasks.workunit.client.0.smithi086.stdout:checking dependency style of gcc... (cached) gcc3 2024-03-22T19:12:32.125 INFO:tasks.workunit.client.0.smithi086.stdout:checking for main in -lm... yes 2024-03-22T19:12:32.198 INFO:tasks.workunit.client.0.smithi086.stdout:checking for main in -lpthread... yes 2024-03-22T19:12:32.200 INFO:tasks.workunit.client.0.smithi086.stdout:checking for ANSI C header files... (cached) yes 2024-03-22T19:12:32.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 -- 172.21.15.86:0/1078155357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42081028d0 msgr2=0x7f4208109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 --2- 172.21.15.86:0/1078155357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42081028d0 0x7f4208109180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f420000b0d0 tx=0x7f420000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 -- 172.21.15.86:0/1078155357 shutdown_connections 2024-03-22T19:12:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 --2- 172.21.15.86:0/1078155357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42081028d0 0x7f4208109180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 --2- 172.21.15.86:0/1078155357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4208100d40 0x7f4208101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 --2- 172.21.15.86:0/1078155357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208101f30 0x7f4208102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 -- 172.21.15.86:0/1078155357 >> 172.21.15.86:0/1078155357 conn(0x7f42080fc2b0 msgr2=0x7f42080fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 -- 172.21.15.86:0/1078155357 shutdown_connections 2024-03-22T19:12:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.252+0000 7f420fc00700 1 -- 172.21.15.86:0/1078155357 wait complete. 2024-03-22T19:12:32.255 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.254+0000 7f420fc00700 1 Processor -- start 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.254+0000 7f420fc00700 1 -- start start 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.254+0000 7f420fc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208100d40 0x7f4208198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.254+0000 7f420fc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 0x7f4208199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.254+0000 7f420fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42081028d0 0x7f42081a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081a0b20 con 0x7f42081028d0 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420fc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081a0c60 con 0x7f4208101f30 2024-03-22T19:12:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420fc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081a0df0 con 0x7f4208100d40 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 0x7f4208199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 0x7f4208199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34652/0 (socket says 172.21.15.86:34652) 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 -- 172.21.15.86:0/2318030117 learned_addr learned my addr 172.21.15.86:0/2318030117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420dc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42081028d0 0x7f42081a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420d200700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208100d40 0x7f4208198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 -- 172.21.15.86:0/2318030117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208100d40 msgr2=0x7f4208198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208100d40 0x7f4208198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 -- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42081028d0 msgr2=0x7f42081a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42081028d0 0x7f42081a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 -- 172.21.15.86:0/2318030117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc009520 con 0x7f4208101f30 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420d200700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208100d40 0x7f4208198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f420dc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42081028d0 0x7f42081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.255+0000 7f4207e00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 0x7f4208199530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e9a0 tx=0x7f41f400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.256+0000 7f4205600700 1 -- 172.21.15.86:0/2318030117 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4009790 con 0x7f4208101f30 2024-03-22T19:12:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.256+0000 7f4205600700 1 -- 172.21.15.86:0/2318030117 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41f400cbd0 con 0x7f4208101f30 2024-03-22T19:12:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.256+0000 7f4205600700 1 -- 172.21.15.86:0/2318030117 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4018680 con 0x7f4208101f30 2024-03-22T19:12:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.256+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4200009d00 con 0x7f4208101f30 2024-03-22T19:12:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.256+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42081a1450 con 0x7f4208101f30 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.259+0000 7f4205600700 1 -- 172.21.15.86:0/2318030117 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f41f400cd40 con 0x7f4208101f30 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.260+0000 7f4223794b80 1 Processor -- start 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.260+0000 7f4205600700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41f00a0b10 0x7f41f00a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.260+0000 7f4223794b80 1 -- start start 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.260+0000 7f4205600700 1 -- 172.21.15.86:0/2318030117 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f41f4014070 con 0x7f4208101f30 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f420dc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41f00a0b10 0x7f41f00a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4223794b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4223794b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44eb580 0x55e6d44ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4223794b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44ec420 0x55e6d44ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4223794b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4443990 con 0x55e6d44ec420 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4223794b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4441350 con 0x55e6d44eac20 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4223794b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d443cc40 con 0x55e6d44eb580 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34660/0 (socket says 172.21.15.86:34660) 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f420d200700 1 -- 172.21.15.86:0/3170966496 learned_addr learned my addr 172.21.15.86:0/3170966496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f4207e00700 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44eb580 0x55e6d44ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.261+0000 7f420dc00700 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44ec420 0x55e6d44ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420d200700 1 -- 172.21.15.86:0/3170966496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44eb580 msgr2=0x55e6d44ebe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420d200700 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44eb580 0x55e6d44ebe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420d200700 1 -- 172.21.15.86:0/3170966496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44ec420 msgr2=0x55e6d44ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420d200700 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44ec420 0x55e6d44ec8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420d200700 1 -- 172.21.15.86:0/3170966496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6d42be3b0 con 0x55e6d44eac20 2024-03-22T19:12:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420dc00700 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44ec420 0x55e6d44ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:12:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420dc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41f00a0b10 0x7f41f00a2fd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4200009fd0 tx=0x7f4200009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.262+0000 7f420d200700 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44eb040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009f10 tx=0x7f41fc0116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4651382a633908dd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.265+0000 7f41fa000700 1 -- 172.21.15.86:0/3170966496 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011ee0 con 0x55e6d44eac20 2024-03-22T19:12:32.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.265+0000 7f41fa000700 1 -- 172.21.15.86:0/3170966496 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41fc012910 con 0x55e6d44eac20 2024-03-22T19:12:32.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.265+0000 7f41fa000700 1 -- 172.21.15.86:0/3170966496 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00f9a0 con 0x55e6d44eac20 2024-03-22T19:12:32.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 -- 172.21.15.86:0/3170966496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 msgr2=0x55e6d44eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44eb040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009f10 tx=0x7f41fc0116c0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 -- 172.21.15.86:0/3170966496 shutdown_connections 2024-03-22T19:12:32.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44ec420 0x55e6d44ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44eb040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 --2- 172.21.15.86:0/3170966496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44eb580 0x55e6d44ebe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 -- 172.21.15.86:0/3170966496 >> 172.21.15.86:0/3170966496 conn(0x55e6d4460220 msgr2=0x55e6d44e5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 -- 172.21.15.86:0/3170966496 shutdown_connections 2024-03-22T19:12:32.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 -- 172.21.15.86:0/3170966496 wait complete. 2024-03-22T19:12:32.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.266+0000 7f4223794b80 1 -- start start 2024-03-22T19:12:32.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.267+0000 7f4223794b80 1 Processor -- start 2024-03-22T19:12:32.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.267+0000 7f4223794b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.267+0000 7f4223794b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f4223794b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f4223794b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d43d8d80 con 0x55e6d44eb580 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f4223794b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d43d8a20 con 0x55e6d44eac20 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f4223794b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d42c06a0 con 0x55e6d44ec420 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f4223794b80 10 client.0 _ll_register_callbacks cb 0x55e6d42443a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34676/0 (socket says 172.21.15.86:34676) 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 -- 172.21.15.86:0/1171786716 learned_addr learned my addr 172.21.15.86:0/1171786716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f4207e00700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420dc00700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 -- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 msgr2=0x55e6d44fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44fc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 -- 172.21.15.86:0/1171786716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 msgr2=0x55e6d44f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.268+0000 7f420d200700 1 -- 172.21.15.86:0/1171786716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc009b40 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.269+0000 7f420d200700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f4ac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41fc012f90 tx=0x7f41fc012df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b6e2a3e4d9c836 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.269+0000 7f41eea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f41eea00700 1 -- 172.21.15.86:0/1171786716 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc004020 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f41eea00700 1 -- 172.21.15.86:0/1171786716 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f41fc004180 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6d44ddad0 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f41eea00700 1 -- 172.21.15.86:0/1171786716 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41fc004340 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f4223794b80 10 client.25757 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6d426ab60 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f41eea00700 1 -- 172.21.15.86:0/1171786716 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6d44ddad0 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.270+0000 7f41eea00700 1 -- 172.21.15.86:0/1171786716 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f41fc022040 con 0x55e6d44eac20 2024-03-22T19:12:32.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4223794b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:32.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4223794b80 10 client.25757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:12:32.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4223794b80 20 client.25757 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:32.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4223794b80 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4500710 0x55e6d4502b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4223794b80 4 client.25757 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:32.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6d4435700 con 0x55e6d4500710 2024-03-22T19:12:32.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.271+0000 7f4207e00700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4500710 0x55e6d4502b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.272+0000 7f4207e00700 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4500710 0x55e6d4502b30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26599951ef5bf257 server_cookie=6930da8467f2abf2 in_seq=0 out_seq=0 2024-03-22T19:12:32.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.272+0000 7f41eea00700 10 client.25757 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.272+0000 7f41eea00700 1 -- 172.21.15.86:0/1171786716 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e6d4435700 con 0x55e6d4500710 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.272+0000 7f41eea00700 10 client.25757 handle_command_reply: tid=0 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 client.25757 shutdown 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 msgr2=0x55e6d44f4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f4ac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41fc012f90 tx=0x7f41fc012df0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 shutdown_connections 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4500710 0x55e6d4502b30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f41eea00700 0 client.25757 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f4ac0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 --2- 172.21.15.86:0/1171786716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 >> 172.21.15.86:0/1171786716 conn(0x55e6d4460220 msgr2=0x55e6d44e51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 shutdown_connections 2024-03-22T19:12:32.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 1 -- 172.21.15.86:0/1171786716 wait complete. 2024-03-22T19:12:32.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.273+0000 7f4223794b80 20 client.25757 trim_cache size 0 max 16384 2024-03-22T19:12:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 Processor -- start 2024-03-22T19:12:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 -- start start 2024-03-22T19:12:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d4507470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d44dff80 con 0x55e6d44eb580 2024-03-22T19:12:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d44e6930 con 0x55e6d44eac20 2024-03-22T19:12:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4223794b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d44ebc30 con 0x55e6d44ec420 2024-03-22T19:12:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d4507470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d4507470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34692/0 (socket says 172.21.15.86:34692) 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f4207e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.288+0000 7f420d200700 1 -- 172.21.15.86:0/1188434558 learned_addr learned my addr 172.21.15.86:0/1188434558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420dc00700 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420d200700 1 -- 172.21.15.86:0/1188434558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 msgr2=0x55e6d44ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420d200700 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420d200700 1 -- 172.21.15.86:0/1188434558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 msgr2=0x55e6d44f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420d200700 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420d200700 1 -- 172.21.15.86:0/1188434558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6d4402ea0 con 0x55e6d44eac20 2024-03-22T19:12:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f420d200700 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d4507470 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41fc012010 tx=0x7f41fc0116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f8176339b009fe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f41faa00700 1 -- 172.21.15.86:0/1188434558 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc012450 con 0x55e6d44eac20 2024-03-22T19:12:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f41faa00700 1 -- 172.21.15.86:0/1188434558 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41fc004bd0 con 0x55e6d44eac20 2024-03-22T19:12:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.289+0000 7f41faa00700 1 -- 172.21.15.86:0/1188434558 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0239e0 con 0x55e6d44eac20 2024-03-22T19:12:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 -- 172.21.15.86:0/1188434558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 msgr2=0x55e6d4507470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d4507470 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41fc012010 tx=0x7f41fc0116c0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 -- 172.21.15.86:0/1188434558 shutdown_connections 2024-03-22T19:12:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d4507470 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 --2- 172.21.15.86:0/1188434558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 -- 172.21.15.86:0/1188434558 >> 172.21.15.86:0/1188434558 conn(0x55e6d4460220 msgr2=0x55e6d44f3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 -- 172.21.15.86:0/1188434558 shutdown_connections 2024-03-22T19:12:32.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.290+0000 7f4223794b80 1 -- 172.21.15.86:0/1188434558 wait complete. 2024-03-22T19:12:32.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 -- start start 2024-03-22T19:12:32.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 Processor -- start 2024-03-22T19:12:32.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d44f9560 con 0x55e6d44eb580 2024-03-22T19:12:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d44f96d0 con 0x55e6d44eac20 2024-03-22T19:12:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d44f9890 con 0x55e6d44ec420 2024-03-22T19:12:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.291+0000 7f4223794b80 10 client.0 _ll_register_callbacks cb 0x7f41f4095d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f4207e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34700/0 (socket says 172.21.15.86:34700) 2024-03-22T19:12:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 -- 172.21.15.86:0/3375135574 learned_addr learned my addr 172.21.15.86:0/3375135574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420dc00700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 -- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 msgr2=0x55e6d44f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 -- 172.21.15.86:0/3375135574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 msgr2=0x55e6d44f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 -- 172.21.15.86:0/3375135574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6d426ab60 con 0x55e6d44eac20 2024-03-22T19:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f4207e00700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f420d200700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f8190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41fc005fe0 tx=0x7f41fc0239b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c63852f820123d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.292+0000 7f41eea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f41eea00700 1 -- 172.21.15.86:0/3375135574 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc012450 con 0x55e6d44eac20 2024-03-22T19:12:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41fc012910 con 0x55e6d44eac20 2024-03-22T19:12:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f41eea00700 1 -- 172.21.15.86:0/3375135574 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f41fc000e40 con 0x55e6d44eac20 2024-03-22T19:12:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f41eea00700 1 -- 172.21.15.86:0/3375135574 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41fc0044e0 con 0x55e6d44eac20 2024-03-22T19:12:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 10 client.25763 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6d42c2cd0 con 0x55e6d44eac20 2024-03-22T19:12:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f41eea00700 1 -- 172.21.15.86:0/3375135574 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41fc012910 con 0x55e6d44eac20 2024-03-22T19:12:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f41eea00700 1 -- 172.21.15.86:0/3375135574 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f41fc03e070 con 0x55e6d44eac20 2024-03-22T19:12:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 10 client.25763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:12:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 20 client.25763 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4552a70 0x55e6d4554e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 4 client.25763 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:12:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.293+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6d428bcf0 con 0x55e6d4552a70 2024-03-22T19:12:32.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.294+0000 7f4207e00700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4552a70 0x55e6d4554e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.294+0000 7f41eea00700 10 client.25763 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:12:32.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.294+0000 7f4207e00700 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4552a70 0x55e6d4554e60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97136313975004a5 server_cookie=233886381b061b9d in_seq=0 out_seq=0 2024-03-22T19:12:32.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.294+0000 7f41eea00700 1 -- 172.21.15.86:0/3375135574 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6d428bcf0 con 0x55e6d4552a70 2024-03-22T19:12:32.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.294+0000 7f41eea00700 10 client.25763 handle_command_reply: tid=0 2024-03-22T19:12:32.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.294+0000 7f4223794b80 1 client.25763 shutdown 2024-03-22T19:12:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 msgr2=0x55e6d44f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f8190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41fc005fe0 tx=0x7f41fc0239b0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 shutdown_connections 2024-03-22T19:12:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55e6d4552a70 0x55e6d4554e60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6d44ec420 0x55e6d44f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f41eea00700 0 client.25763 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:12:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6d44eac20 0x55e6d44f8190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 --2- 172.21.15.86:0/3375135574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6d44eb580 0x55e6d44f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 >> 172.21.15.86:0/3375135574 conn(0x55e6d4460220 msgr2=0x55e6d44e5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 shutdown_connections 2024-03-22T19:12:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 1 -- 172.21.15.86:0/3375135574 wait complete. 2024-03-22T19:12:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.295+0000 7f4223794b80 20 client.25763 trim_cache size 0 max 16384 2024-03-22T19:12:32.312 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:12:32.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41f00a0b10 msgr2=0x7f41f00a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41f00a0b10 0x7f41f00a2fd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4200009fd0 tx=0x7f4200009500 comp rx=0 tx=0).stop 2024-03-22T19:12:32.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 msgr2=0x7f4208199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 0x7f4208199530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e9a0 tx=0x7f41f400ed60 comp rx=0 tx=0).stop 2024-03-22T19:12:32.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 shutdown_connections 2024-03-22T19:12:32.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f41f00a0b10 0x7f41f00a2fd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42081028d0 0x7f42081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4208100d40 0x7f4208198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 --2- 172.21.15.86:0/2318030117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4208101f30 0x7f4208199530 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 >> 172.21.15.86:0/2318030117 conn(0x7f42080fc2b0 msgr2=0x7f4208072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 shutdown_connections 2024-03-22T19:12:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.296+0000 7f420fc00700 1 -- 172.21.15.86:0/2318030117 wait complete. 2024-03-22T19:12:32.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:12:32.350 INFO:tasks.workunit.client.0.smithi086.stdout:checking pthread.h usability... yes 2024-03-22T19:12:32.376 INFO:tasks.workunit.client.0.smithi086.stdout:checking pthread.h presence... yes 2024-03-22T19:12:32.376 INFO:tasks.workunit.client.0.smithi086.stdout:checking for pthread.h... yes 2024-03-22T19:12:32.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:32 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2203446429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:12:32.454 INFO:tasks.workunit.client.0.smithi086.stdout:checking fcntl.h usability... yes 2024-03-22T19:12:32.477 INFO:tasks.workunit.client.0.smithi086.stdout:checking fcntl.h presence... yes 2024-03-22T19:12:32.477 INFO:tasks.workunit.client.0.smithi086.stdout:checking for fcntl.h... yes 2024-03-22T19:12:32.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 -- 172.21.15.86:0/1820731427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 msgr2=0x7fa8e80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 --2- 172.21.15.86:0/1820731427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e80731c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009990 tx=0x7fa8d8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 -- 172.21.15.86:0/1820731427 shutdown_connections 2024-03-22T19:12:32.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 --2- 172.21.15.86:0/1820731427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8e8108c40 0x7fa8e810b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 --2- 172.21.15.86:0/1820731427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e80731c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 --2- 172.21.15.86:0/1820731427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 0x7fa8e8073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 -- 172.21.15.86:0/1820731427 >> 172.21.15.86:0/1820731427 conn(0x7fa8e80fc2c0 msgr2=0x7fa8e80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 -- 172.21.15.86:0/1820731427 shutdown_connections 2024-03-22T19:12:32.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.528+0000 7fa8ee600700 1 -- 172.21.15.86:0/1820731427 wait complete. 2024-03-22T19:12:32.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 Processor -- start 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 -- start start 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 0x7fa8e8198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8e8108c40 0x7fa8e81a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81a0990 con 0x7fa8e8074d70 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81a0ad0 con 0x7fa8e8073790 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8ee600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81a0c60 con 0x7fa8e8108c40 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.530+0000 7fa8e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32830/0 (socket says 172.21.15.86:32830) 2024-03-22T19:12:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 -- 172.21.15.86:0/946606701 learned_addr learned my addr 172.21.15.86:0/946606701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e7400700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 0x7fa8e8198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8e8108c40 0x7fa8e81a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 -- 172.21.15.86:0/946606701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8e8108c40 msgr2=0x7fa8e81a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8e8108c40 0x7fa8e81a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 -- 172.21.15.86:0/946606701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 msgr2=0x7fa8e8198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 0x7fa8e8198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 -- 172.21.15.86:0/946606701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009520 con 0x7fa8e8074d70 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e7400700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 0x7fa8e8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8e6a00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e81993f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400eb30 tx=0x7fa8d400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8dfe00700 1 -- 172.21.15.86:0/946606701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4009880 con 0x7fa8e8074d70 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8dfe00700 1 -- 172.21.15.86:0/946606701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8d40044e0 con 0x7fa8e8074d70 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.531+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d80095f0 con 0x7fa8e8074d70 2024-03-22T19:12:32.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.532+0000 7fa8dfe00700 1 -- 172.21.15.86:0/946606701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4003d80 con 0x7fa8e8074d70 2024-03-22T19:12:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.532+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e81a1260 con 0x7fa8e8074d70 2024-03-22T19:12:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.535+0000 7fa8dfe00700 1 -- 172.21.15.86:0/946606701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa8d4004000 con 0x7fa8e8074d70 2024-03-22T19:12:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 Processor -- start 2024-03-22T19:12:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 -- start start 2024-03-22T19:12:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 0x55764aa65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa65e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa8dfe00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8cc0a0a90 0x7fa8cc0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa66420 0x55764aa668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764a9bd990 con 0x55764aa64c20 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.536+0000 7fa902167b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764a9bb350 con 0x55764aa66420 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa902167b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764a9b6c40 con 0x55764aa65580 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8dfe00700 1 -- 172.21.15.86:0/946606701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8d4014070 con 0x7fa8e8074d70 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 0x55764aa65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 0x55764aa65040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32836/0 (socket says 172.21.15.86:32836) 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 -- 172.21.15.86:0/3341541609 learned_addr learned my addr 172.21.15.86:0/3341541609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7400700 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa66420 0x55764aa668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e6a00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8cc0a0a90 0x7fa8cc0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 -- 172.21.15.86:0/3341541609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 msgr2=0x55764aa65e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa65e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 -- 172.21.15.86:0/3341541609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa66420 msgr2=0x55764aa668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa66420 0x55764aa668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7e00700 1 -- 172.21.15.86:0/3341541609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55764a8383b0 con 0x55764aa64c20 2024-03-22T19:12:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.537+0000 7fa8e7400700 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa65e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.538+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 0x55764aa65040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009690 tx=0x7fa8e0003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b536cb896c00f951 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.538+0000 7fa8e6a00700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8cc0a0a90 0x7fa8cc0a2f50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4000c00 tx=0x7fa8d4003820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.538+0000 7fa8dcc00700 1 -- 172.21.15.86:0/3341541609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004db0 con 0x55764aa64c20 2024-03-22T19:12:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.538+0000 7fa8dcc00700 1 -- 172.21.15.86:0/3341541609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8e000f040 con 0x55764aa64c20 2024-03-22T19:12:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.538+0000 7fa8dcc00700 1 -- 172.21.15.86:0/3341541609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e001b6c0 con 0x55764aa64c20 2024-03-22T19:12:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 -- 172.21.15.86:0/3341541609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 msgr2=0x55764aa65040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 0x55764aa65040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009690 tx=0x7fa8e0003ac0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 -- 172.21.15.86:0/3341541609 shutdown_connections 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa66420 0x55764aa668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa64c20 0x55764aa65040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 --2- 172.21.15.86:0/3341541609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa65e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 -- 172.21.15.86:0/3341541609 >> 172.21.15.86:0/3341541609 conn(0x55764a9da220 msgr2=0x55764aa5fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 -- 172.21.15.86:0/3341541609 shutdown_connections 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 -- 172.21.15.86:0/3341541609 wait complete. 2024-03-22T19:12:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.539+0000 7fa902167b80 1 -- start start 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 Processor -- start 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa6ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa6ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa75fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764a952d80 con 0x55764aa66420 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764a952a20 con 0x55764aa65580 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764a83a6a0 con 0x55764aa64c20 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa902167b80 10 client.0 _ll_register_callbacks cb 0x55764a7be3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.540+0000 7fa8e7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa6ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa6ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51400/0 (socket says 172.21.15.86:51400) 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7e00700 1 -- 172.21.15.86:0/1733701107 learned_addr learned my addr 172.21.15.86:0/1733701107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa6ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa75fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 -- 172.21.15.86:0/1733701107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 msgr2=0x55764aa6ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa6ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 -- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 msgr2=0x55764aa75fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa75fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 -- 172.21.15.86:0/1733701107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e810bcc0 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8e7400700 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa6ef80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b7b0 tx=0x7fa8d800bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756474f956e73d48 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.541+0000 7fa8d1600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa8d1600700 1 -- 172.21.15.86:0/1733701107 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d801d070 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa8d1600700 1 -- 172.21.15.86:0/1733701107 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8d800fd10 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55764aa57ad0 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa8d1600700 1 -- 172.21.15.86:0/1733701107 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d8024450 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa902167b80 10 client.25775 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa8e0004040 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa8d1600700 1 -- 172.21.15.86:0/1733701107 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55764aa57ad0 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa8d1600700 1 -- 172.21.15.86:0/1733701107 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8d8017580 con 0x55764aa65580 2024-03-22T19:12:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa902167b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.542+0000 7fa902167b80 10 client.25775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa902167b80 20 client.25775 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa902167b80 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aa7a890 0x55764aa7ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa902167b80 4 client.25775 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55764a9af700 con 0x55764aa7a890 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aa7a890 0x55764aa7ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aa7a890 0x55764aa7ccb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=781fda3885e1c14b server_cookie=190b130067ed1afc in_seq=0 out_seq=0 2024-03-22T19:12:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.543+0000 7fa8d1600700 10 client.25775 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa8d1600700 1 -- 172.21.15.86:0/1733701107 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55764a9af700 con 0x55764aa7a890 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa8d1600700 10 client.25775 handle_command_reply: tid=0 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 client.25775 shutdown 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 msgr2=0x55764aa6ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa6ef80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b7b0 tx=0x7fa8d800bf70 comp rx=0 tx=0).stop 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 shutdown_connections 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aa7a890 0x55764aa7ccb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa75fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa8d1600700 0 client.25775 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:12:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa6ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 --2- 172.21.15.86:0/1733701107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa6ef80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 >> 172.21.15.86:0/1733701107 conn(0x55764a9da220 msgr2=0x55764aa5f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 shutdown_connections 2024-03-22T19:12:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.544+0000 7fa902167b80 1 -- 172.21.15.86:0/1733701107 wait complete. 2024-03-22T19:12:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.545+0000 7fa902167b80 20 client.25775 trim_cache size 0 max 16384 2024-03-22T19:12:32.560 INFO:tasks.workunit.client.0.smithi086.stdout:checking limits.h usability... yes 2024-03-22T19:12:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 Processor -- start 2024-03-22T19:12:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 -- start start 2024-03-22T19:12:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa64c20 0x55764aa73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa6df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa76670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764aa819d0 con 0x55764aa66420 2024-03-22T19:12:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764aa66050 con 0x55764aa64c20 2024-03-22T19:12:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.562+0000 7fa902167b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764aa6fc30 con 0x55764aa65580 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa6df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa64c20 0x55764aa73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa6df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51416/0 (socket says 172.21.15.86:51416) 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 -- 172.21.15.86:0/548431940 learned_addr learned my addr 172.21.15.86:0/548431940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa76670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 -- 172.21.15.86:0/548431940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa64c20 msgr2=0x55764aa73f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa64c20 0x55764aa73f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 -- 172.21.15.86:0/548431940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 msgr2=0x55764aa76670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa76670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 -- 172.21.15.86:0/548431940 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55764a7e4b60 con 0x55764aa65580 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.563+0000 7fa8e7400700 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa6df50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80098e0 tx=0x7fa8d800b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e434f12f8ebcc273 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa8dd600700 1 -- 172.21.15.86:0/548431940 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d801d070 con 0x55764aa65580 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa8dd600700 1 -- 172.21.15.86:0/548431940 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8d801e040 con 0x55764aa65580 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa8dd600700 1 -- 172.21.15.86:0/548431940 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8024980 con 0x55764aa65580 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 -- 172.21.15.86:0/548431940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 msgr2=0x55764aa6df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa6df50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80098e0 tx=0x7fa8d800b2c0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 -- 172.21.15.86:0/548431940 shutdown_connections 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa64c20 0x55764aa73f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 --2- 172.21.15.86:0/548431940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa65580 0x55764aa6df50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.564+0000 7fa902167b80 1 -- 172.21.15.86:0/548431940 >> 172.21.15.86:0/548431940 conn(0x55764a9da220 msgr2=0x55764aa6c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.565+0000 7fa902167b80 1 -- 172.21.15.86:0/548431940 shutdown_connections 2024-03-22T19:12:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.565+0000 7fa902167b80 1 -- 172.21.15.86:0/548431940 wait complete. 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.565+0000 7fa902167b80 1 -- start start 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.565+0000 7fa902167b80 1 Processor -- start 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa71ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa73230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764aa73770 con 0x55764aa66420 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764aa738e0 con 0x55764aa65580 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764aa73aa0 con 0x55764aa64c20 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa902167b80 10 client.0 _ll_register_callbacks cb 0x55764a7be3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa72220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34760/0 (socket says 172.21.15.86:34760) 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 -- 172.21.15.86:0/2975968075 learned_addr learned my addr 172.21.15.86:0/2975968075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa73230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa71ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 -- 172.21.15.86:0/2975968075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 msgr2=0x55764aa71ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa71ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 -- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 msgr2=0x55764aa73230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa73230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7400700 1 -- 172.21.15.86:0/2975968075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e0004040 con 0x55764aa65580 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.566+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa71ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8e7400700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa72220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800bd10 tx=0x7fa8d800bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6571a27479394bd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8d1600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8d1600700 1 -- 172.21.15.86:0/2975968075 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d801d070 con 0x55764aa65580 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8e000f040 con 0x55764aa65580 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8d1600700 1 -- 172.21.15.86:0/2975968075 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8d800fb10 con 0x55764aa65580 2024-03-22T19:12:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8d1600700 1 -- 172.21.15.86:0/2975968075 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d800fcd0 con 0x55764aa65580 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa902167b80 10 client.25781 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55764a97cea0 con 0x55764aa65580 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8d1600700 1 -- 172.21.15.86:0/2975968075 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8d804f020 con 0x55764aa65580 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.567+0000 7fa8d1600700 1 -- 172.21.15.86:0/2975968075 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8d804b750 con 0x55764aa65580 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa902167b80 10 client.25781 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa902167b80 10 client.25781 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa902167b80 20 client.25781 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa902167b80 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aaccec0 0x55764aacf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa902167b80 4 client.25781 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55764a805cf0 con 0x55764aaccec0 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aaccec0 0x55764aacf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa8d1600700 10 client.25781 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:12:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.568+0000 7fa8e7e00700 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aaccec0 0x55764aacf2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36a0013f2ae49630 server_cookie=33cb05bedca11ea3 in_seq=0 out_seq=0 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa8d1600700 1 -- 172.21.15.86:0/2975968075 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55764a805cf0 con 0x55764aaccec0 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa8d1600700 10 client.25781 handle_command_reply: tid=0 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 client.25781 shutdown 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 msgr2=0x55764aa72220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa72220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800bd10 tx=0x7fa8d800bf50 comp rx=0 tx=0).stop 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 shutdown_connections 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55764aaccec0 0x55764aacf2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55764aa66420 0x55764aa73230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55764aa64c20 0x55764aa71ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 --2- 172.21.15.86:0/2975968075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55764aa65580 0x55764aa72220 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa8d1600700 0 client.25781 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 >> 172.21.15.86:0/2975968075 conn(0x55764a9da220 msgr2=0x55764aa6c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 shutdown_connections 2024-03-22T19:12:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 1 -- 172.21.15.86:0/2975968075 wait complete. 2024-03-22T19:12:32.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.569+0000 7fa902167b80 20 client.25781 trim_cache size 0 max 16384 2024-03-22T19:12:32.574 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8cc0a0a90 msgr2=0x7fa8cc0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8cc0a0a90 0x7fa8cc0a2f50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4000c00 tx=0x7fa8d4003820 comp rx=0 tx=0).stop 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 msgr2=0x7fa8e81993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e81993f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400eb30 tx=0x7fa8d400ee40 comp rx=0 tx=0).stop 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 shutdown_connections 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8cc0a0a90 0x7fa8cc0a2f50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8e8108c40 0x7fa8e81a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8073790 0x7fa8e8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 --2- 172.21.15.86:0/946606701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e8074d70 0x7fa8e81993f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 >> 172.21.15.86:0/946606701 conn(0x7fa8e80fc2c0 msgr2=0x7fa8e8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 shutdown_connections 2024-03-22T19:12:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.570+0000 7fa8ee600700 1 -- 172.21.15.86:0/946606701 wait complete. 2024-03-22T19:12:32.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:32 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2203446429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:12:32.584 INFO:tasks.workunit.client.0.smithi086.stdout:checking limits.h presence... yes 2024-03-22T19:12:32.584 INFO:tasks.workunit.client.0.smithi086.stdout:checking for limits.h... yes 2024-03-22T19:12:32.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:12:32.607 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stdint.h... (cached) yes 2024-03-22T19:12:32.676 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/time.h usability... yes 2024-03-22T19:12:32.709 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/time.h presence... yes 2024-03-22T19:12:32.709 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/time.h... yes 2024-03-22T19:12:32.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:32 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2203446429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:12:32.739 INFO:tasks.workunit.client.0.smithi086.stdout:checking for unistd.h... (cached) yes 2024-03-22T19:12:32.814 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/vfs.h usability... yes 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 -- 172.21.15.86:0/2588775363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c0100c60 msgr2=0x7fe6c0101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/2588775363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c0100c60 0x7fe6c0101080 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00099c0 tx=0x7fe6b0009cd0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 -- 172.21.15.86:0/2588775363 shutdown_connections 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/2588775363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c01027f0 0x7fe6c01090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/2588775363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c0100c60 0x7fe6c0101080 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/2588775363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c0101e50 0x7fe6c01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 -- 172.21.15.86:0/2588775363 >> 172.21.15.86:0/2588775363 conn(0x7fe6c00fc1f0 msgr2=0x7fe6c00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 -- 172.21.15.86:0/2588775363 shutdown_connections 2024-03-22T19:12:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.820+0000 7fe6c5c00700 1 -- 172.21.15.86:0/2588775363 wait complete. 2024-03-22T19:12:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 Processor -- start 2024-03-22T19:12:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 -- start start 2024-03-22T19:12:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 0x7fe6c0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c0101e50 0x7fe6c00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c01027f0 0x7fe6c00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0073e80 con 0x7fe6c01027f0 2024-03-22T19:12:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0073ff0 con 0x7fe6c0100c60 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6c5c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01a0c70 con 0x7fe6c0101e50 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 0x7fe6c0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 0x7fe6c0074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34782/0 (socket says 172.21.15.86:34782) 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6bea00700 1 -- 172.21.15.86:0/1538773329 learned_addr learned my addr 172.21.15.86:0/1538773329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6bf400700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c01027f0 0x7fe6c00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.822+0000 7fe6be000700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c0101e50 0x7fe6c00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6bea00700 1 -- 172.21.15.86:0/1538773329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c0101e50 msgr2=0x7fe6c00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6bea00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c0101e50 0x7fe6c00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6bea00700 1 -- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c01027f0 msgr2=0x7fe6c00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6bea00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c01027f0 0x7fe6c00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6bea00700 1 -- 172.21.15.86:0/1538773329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00095f0 con 0x7fe6c0100c60 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6bea00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 0x7fe6c0074b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006010 tx=0x7fe6b000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6b7400700 1 -- 172.21.15.86:0/1538773329 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b001d070 con 0x7fe6c0100c60 2024-03-22T19:12:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6b7400700 1 -- 172.21.15.86:0/1538773329 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b0005020 con 0x7fe6c0100c60 2024-03-22T19:12:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c01a0ea0 con 0x7fe6c0100c60 2024-03-22T19:12:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.823+0000 7fe6b7400700 1 -- 172.21.15.86:0/1538773329 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0004b10 con 0x7fe6c0100c60 2024-03-22T19:12:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.824+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c01a1210 con 0x7fe6c0100c60 2024-03-22T19:12:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.827+0000 7fe6b7400700 1 -- 172.21.15.86:0/1538773329 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe6b0005190 con 0x7fe6c0100c60 2024-03-22T19:12:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 Processor -- start 2024-03-22T19:12:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6b7400700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe6a40a0a80 0x7fe6a40a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 -- start start 2024-03-22T19:12:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6b7400700 1 -- 172.21.15.86:0/1538773329 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe6b00ccaa0 con 0x7fe6c0100c60 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9d96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6bf400700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe6a40a0a80 0x7fe6a40a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d96580 0x55cbe9d96e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9d978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9cee990 con 0x55cbe9d97420 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9cec350 con 0x55cbe9d96580 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.829+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9ce7c40 con 0x55cbe9d95c20 2024-03-22T19:12:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9d96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9d96040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51454/0 (socket says 172.21.15.86:51454) 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 -- 172.21.15.86:0/3300503658 learned_addr learned my addr 172.21.15.86:0/3300503658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bf400700 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9d978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6be000700 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d96580 0x55cbe9d96e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 -- 172.21.15.86:0/3300503658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d96580 msgr2=0x55cbe9d96e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d96580 0x55cbe9d96e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 -- 172.21.15.86:0/3300503658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 msgr2=0x55cbe9d978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9d978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 -- 172.21.15.86:0/3300503658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe9b693b0 con 0x55cbe9d95c20 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9d96040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00966b0 tx=0x7fe6b0099b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7686e8744bf49100 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.830+0000 7fe6bf400700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe6a40a0a80 0x7fe6a40a2f40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800f870 tx=0x7fe6b800fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6abe00700 1 -- 172.21.15.86:0/3300503658 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00850c0 con 0x55cbe9d95c20 2024-03-22T19:12:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6abe00700 1 -- 172.21.15.86:0/3300503658 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b009dfe0 con 0x55cbe9d95c20 2024-03-22T19:12:32.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6abe00700 1 -- 172.21.15.86:0/3300503658 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b009a410 con 0x55cbe9d95c20 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3300503658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 msgr2=0x55cbe9d96040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9d96040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00966b0 tx=0x7fe6b0099b90 comp rx=0 tx=0).stop 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3300503658 shutdown_connections 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9d978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9d96040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3300503658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d96580 0x55cbe9d96e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.831+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3300503658 >> 172.21.15.86:0/3300503658 conn(0x55cbe9d0b220 msgr2=0x55cbe9d90bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.832+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3300503658 shutdown_connections 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.832+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3300503658 wait complete. 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.832+0000 7fe6d974ab80 1 -- start start 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.832+0000 7fe6d974ab80 1 Processor -- start 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9d9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d97420 0x55cbe9da6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9c83d80 con 0x55cbe9d96580 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9c83a20 con 0x55cbe9d95c20 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9b6b6a0 con 0x55cbe9d97420 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6d974ab80 10 client.0 _ll_register_callbacks cb 0x55cbe9aef3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6be000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6be000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32906/0 (socket says 172.21.15.86:32906) 2024-03-22T19:12:32.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6be000700 1 -- 172.21.15.86:0/3382063255 learned_addr learned my addr 172.21.15.86:0/3382063255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6bf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d97420 0x55cbe9da6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.833+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9d9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6be000700 1 -- 172.21.15.86:0/3382063255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d97420 msgr2=0x55cbe9da6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6be000700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d97420 0x55cbe9da6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6be000700 1 -- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 msgr2=0x55cbe9d9f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6be000700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9d9f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6be000700 1 -- 172.21.15.86:0/3382063255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c01a1210 con 0x55cbe9d96580 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9d9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6be000700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9fca0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0126b0 tx=0x7fe6ac0129c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5095c1b5eef665 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6a8c00700 1 -- 172.21.15.86:0/3382063255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac017070 con 0x55cbe9d96580 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6a8c00700 1 -- 172.21.15.86:0/3382063255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe6ac004050 con 0x55cbe9d96580 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.834+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbe9d88ad0 con 0x55cbe9d96580 2024-03-22T19:12:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6a8c00700 1 -- 172.21.15.86:0/3382063255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6ac004210 con 0x55cbe9d96580 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 10 client.15756 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cbe9b15b60 con 0x55cbe9d96580 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6a8c00700 1 -- 172.21.15.86:0/3382063255 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbe9d88ad0 con 0x55cbe9d96580 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6a8c00700 1 -- 172.21.15.86:0/3382063255 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe6ac0104c0 con 0x55cbe9d96580 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 20 client.15756 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dab580 0x55cbe9dad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 4 client.15756 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.835+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbe9ce0700 con 0x55cbe9dab580 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.836+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dab580 0x55cbe9dad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.836+0000 7fe6bea00700 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dab580 0x55cbe9dad9a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f08fd8a728c8fe server_cookie=e5455dbfc509d52a in_seq=0 out_seq=0 2024-03-22T19:12:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.836+0000 7fe6a8c00700 10 client.15756 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6a8c00700 1 -- 172.21.15.86:0/3382063255 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cbe9ce0700 con 0x55cbe9dab580 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6a8c00700 10 client.15756 handle_command_reply: tid=0 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 client.15756 shutdown 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 msgr2=0x55cbe9d9fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9fca0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0126b0 tx=0x7fe6ac0129c0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 shutdown_connections 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dab580 0x55cbe9dad9a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d97420 0x55cbe9da6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9d9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/3382063255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9fca0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6a8c00700 0 client.15756 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 >> 172.21.15.86:0/3382063255 conn(0x55cbe9d0b220 msgr2=0x55cbe9d901f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 shutdown_connections 2024-03-22T19:12:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 1 -- 172.21.15.86:0/3382063255 wait complete. 2024-03-22T19:12:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.838+0000 7fe6d974ab80 20 client.15756 trim_cache size 0 max 16384 2024-03-22T19:12:32.842 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/vfs.h presence... yes 2024-03-22T19:12:32.842 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/vfs.h... yes 2024-03-22T19:12:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 Processor -- start 2024-03-22T19:12:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 -- start start 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9da4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d97420 0x55cbe9d97800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9d91930 con 0x55cbe9d96580 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9d8b210 con 0x55cbe9d97420 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9d9f2d0 con 0x55cbe9d95c20 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9da4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6be000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9da4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51464/0 (socket says 172.21.15.86:51464) 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6bea00700 1 -- 172.21.15.86:0/2457103534 learned_addr learned my addr 172.21.15.86:0/2457103534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.852+0000 7fe6bf400700 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d97420 0x55cbe9d97800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6bea00700 1 -- 172.21.15.86:0/2457103534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d97420 msgr2=0x55cbe9d97800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6bea00700 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d97420 0x55cbe9d97800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6bea00700 1 -- 172.21.15.86:0/2457103534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 msgr2=0x55cbe9d9ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6bea00700 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6bea00700 1 -- 172.21.15.86:0/2457103534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe9cadea0 con 0x55cbe9d95c20 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6bea00700 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9da4c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0096c20 tx=0x7fe6b0005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe440aba512c605c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6b4c00700 1 -- 172.21.15.86:0/2457103534 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00b8040 con 0x55cbe9d95c20 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6b4c00700 1 -- 172.21.15.86:0/2457103534 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b00850b0 con 0x55cbe9d95c20 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.853+0000 7fe6b4c00700 1 -- 172.21.15.86:0/2457103534 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b009a410 con 0x55cbe9d95c20 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2457103534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 msgr2=0x55cbe9da4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9da4c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0096c20 tx=0x7fe6b0005ef0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2457103534 shutdown_connections 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d97420 0x55cbe9d97800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d95c20 0x55cbe9da4c90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2457103534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d96580 0x55cbe9d9ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2457103534 >> 172.21.15.86:0/2457103534 conn(0x55cbe9d0b220 msgr2=0x55cbe9d9d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2457103534 shutdown_connections 2024-03-22T19:12:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2457103534 wait complete. 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.854+0000 7fe6d974ab80 1 -- start start 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 Processor -- start 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9da2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d96580 0x55cbe9da2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9da3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9da41a0 con 0x55cbe9d97420 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9da4310 con 0x55cbe9d95c20 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9da44d0 con 0x55cbe9d96580 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6d974ab80 10 client.0 _ll_register_callbacks cb 0x7fe6ac009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9da2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9da2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34806/0 (socket says 172.21.15.86:34806) 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6bea00700 1 -- 172.21.15.86:0/2449301632 learned_addr learned my addr 172.21.15.86:0/2449301632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:32.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6be000700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d96580 0x55cbe9da2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6bf400700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9da3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6bea00700 1 -- 172.21.15.86:0/2449301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d96580 msgr2=0x55cbe9da2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.855+0000 7fe6bea00700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d96580 0x55cbe9da2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6bea00700 1 -- 172.21.15.86:0/2449301632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 msgr2=0x55cbe9da3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6bea00700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9da3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6bea00700 1 -- 172.21.15.86:0/2449301632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbe9b693b0 con 0x55cbe9d95c20 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6bf400700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9da3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6bea00700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9da2710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0085010 tx=0x7fe6b00098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7c6621583a014a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6a8c00700 1 -- 172.21.15.86:0/2449301632 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00b8040 con 0x55cbe9d95c20 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6b009dfe0 con 0x55cbe9d95c20 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6a8c00700 1 -- 172.21.15.86:0/2449301632 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe6b0099910 con 0x55cbe9d95c20 2024-03-22T19:12:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6a8c00700 1 -- 172.21.15.86:0/2449301632 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b00983b0 con 0x55cbe9d95c20 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6d974ab80 10 client.25799 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.856+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cbe9b6dcd0 con 0x55cbe9d95c20 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6a8c00700 1 -- 172.21.15.86:0/2449301632 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b009dfe0 con 0x55cbe9d95c20 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6a8c00700 1 -- 172.21.15.86:0/2449301632 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe6b00b81a0 con 0x55cbe9d95c20 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6d974ab80 10 client.25799 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6d974ab80 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6d974ab80 20 client.25799 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dfd8a0 0x55cbe9dffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6d974ab80 4 client.25799 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbe9b36cf0 con 0x55cbe9dfd8a0 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.857+0000 7fe6be000700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dfd8a0 0x55cbe9dffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6be000700 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dfd8a0 0x55cbe9dffcc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9058454464f872e8 server_cookie=87dcdd13cafb390a in_seq=0 out_seq=0 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6a8c00700 10 client.25799 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6a8c00700 1 -- 172.21.15.86:0/2449301632 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbe9b36cf0 con 0x55cbe9dfd8a0 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6a8c00700 10 client.25799 handle_command_reply: tid=0 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 client.25799 shutdown 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 msgr2=0x55cbe9da2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9da2710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0085010 tx=0x7fe6b00098e0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 shutdown_connections 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55cbe9dfd8a0 0x55cbe9dffcc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbe9d97420 0x55cbe9da3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbe9d95c20 0x55cbe9da2710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6a8c00700 0 client.25799 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 --2- 172.21.15.86:0/2449301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbe9d96580 0x55cbe9da2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.858+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 >> 172.21.15.86:0/2449301632 conn(0x55cbe9d0b220 msgr2=0x55cbe9d9d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 shutdown_connections 2024-03-22T19:12:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6d974ab80 1 -- 172.21.15.86:0/2449301632 wait complete. 2024-03-22T19:12:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6d974ab80 20 client.25799 trim_cache size 0 max 16384 2024-03-22T19:12:32.866 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe6a40a0a80 msgr2=0x7fe6a40a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe6a40a0a80 0x7fe6a40a2f40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800f870 tx=0x7fe6b800fc30 comp rx=0 tx=0).stop 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 msgr2=0x7fe6c0074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 0x7fe6c0074b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006010 tx=0x7fe6b000f690 comp rx=0 tx=0).stop 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 shutdown_connections 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe6a40a0a80 0x7fe6a40a2f40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c01027f0 0x7fe6c00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c0100c60 0x7fe6c0074b10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 --2- 172.21.15.86:0/1538773329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c0101e50 0x7fe6c00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.859+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 >> 172.21.15.86:0/1538773329 conn(0x7fe6c00fc1f0 msgr2=0x7fe6c01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.860+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 shutdown_connections 2024-03-22T19:12:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:32.860+0000 7fe6c5c00700 1 -- 172.21.15.86:0/1538773329 wait complete. 2024-03-22T19:12:32.927 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/limits.h usability... no 2024-03-22T19:12:32.969 INFO:tasks.workunit.client.0.smithi086.stdout:checking sys/limits.h presence... no 2024-03-22T19:12:32.969 INFO:tasks.workunit.client.0.smithi086.stdout:checking for sys/limits.h... no 2024-03-22T19:12:33.013 INFO:tasks.workunit.client.0.smithi086.stdout:checking for an ANSI C-conforming const... yes 2024-03-22T19:12:33.061 INFO:tasks.workunit.client.0.smithi086.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-03-22T19:12:33.155 INFO:tasks.workunit.client.0.smithi086.stdout:checking for system... yes 2024-03-22T19:12:33.249 INFO:tasks.workunit.client.0.smithi086.stdout:checking for gettimeofday... yes 2024-03-22T19:12:33.342 INFO:tasks.workunit.client.0.smithi086.stdout:checking for mkdir... yes 2024-03-22T19:12:33.435 INFO:tasks.workunit.client.0.smithi086.stdout:checking for strerror... yes 2024-03-22T19:12:33.528 INFO:tasks.workunit.client.0.smithi086.stdout:checking for open64... yes 2024-03-22T19:12:33.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:33 smithi167 ceph-mon[47728]: pgmap v333: 97 pgs: 97 active+clean; 4.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-03-22T19:12:33.621 INFO:tasks.workunit.client.0.smithi086.stdout:checking for stat64... yes 2024-03-22T19:12:33.715 INFO:tasks.workunit.client.0.smithi086.stdout:checking for fseeko64... yes 2024-03-22T19:12:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:33 smithi131 ceph-mon[78559]: pgmap v333: 97 pgs: 97 active+clean; 4.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-03-22T19:12:33.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:33 smithi086 ceph-mon[38032]: pgmap v333: 97 pgs: 97 active+clean; 4.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-03-22T19:12:33.809 INFO:tasks.workunit.client.0.smithi086.stdout:checking for lrand48_r... yes 2024-03-22T19:12:33.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:12:33.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:12:33.905 INFO:tasks.workunit.client.0.smithi086.stdout:checking for srand48_r... yes 2024-03-22T19:12:33.949 INFO:tasks.workunit.client.0.smithi086.stdout:configure: creating ./config.status 2024-03-22T19:12:34.067 INFO:tasks.workunit.client.0.smithi086.stdout: /bin/sh ./config.status 2024-03-22T19:12:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 -- 172.21.15.86:0/2196969303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0102f00 msgr2=0x7fcac0103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 --2- 172.21.15.86:0/2196969303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0102f00 0x7fcac0103340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009990 tx=0x7fcab0009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 -- 172.21.15.86:0/2196969303 shutdown_connections 2024-03-22T19:12:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 --2- 172.21.15.86:0/2196969303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0104a90 0x7fcac010b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 --2- 172.21.15.86:0/2196969303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0102f00 0x7fcac0103340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 --2- 172.21.15.86:0/2196969303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 0x7fcac0104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 -- 172.21.15.86:0/2196969303 >> 172.21.15.86:0/2196969303 conn(0x7fcac00fe4c0 msgr2=0x7fcac0100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.087+0000 7fcac6c00700 1 -- 172.21.15.86:0/2196969303 shutdown_connections 2024-03-22T19:12:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.088+0000 7fcac6c00700 1 -- 172.21.15.86:0/2196969303 wait complete. 2024-03-22T19:12:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.088+0000 7fcac6c00700 1 Processor -- start 2024-03-22T19:12:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 -- start start 2024-03-22T19:12:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 0x7fcac0198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 0x7fcac0199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0104a90 0x7fcac01a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a0a50 con 0x7fcac0104a90 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a0b90 con 0x7fcac0102f00 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac6c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a0d20 con 0x7fcac01040d0 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 0x7fcac0198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 0x7fcac0198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34826/0 (socket says 172.21.15.86:34826) 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcabfe00700 1 -- 172.21.15.86:0/3820580893 learned_addr learned my addr 172.21.15.86:0/3820580893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcac4c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0104a90 0x7fcac01a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.089+0000 7fcabf400700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 0x7fcac0199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabfe00700 1 -- 172.21.15.86:0/3820580893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 msgr2=0x7fcac0199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabfe00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 0x7fcac0199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabfe00700 1 -- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0104a90 msgr2=0x7fcac01a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabfe00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0104a90 0x7fcac01a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabfe00700 1 -- 172.21.15.86:0/3820580893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00095f0 con 0x7fcac0102f00 2024-03-22T19:12:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabf400700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 0x7fcac0199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcac4c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0104a90 0x7fcac01a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabfe00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 0x7fcac0198f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcab0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabcc00700 1 -- 172.21.15.86:0/3820580893 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab001d070 con 0x7fcac0102f00 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabcc00700 1 -- 172.21.15.86:0/3820580893 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab0021410 con 0x7fcac0102f00 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac01a0fa0 con 0x7fcac0102f00 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcabcc00700 1 -- 172.21.15.86:0/3820580893 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0017650 con 0x7fcac0102f00 2024-03-22T19:12:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.090+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a1310 con 0x7fcac0102f00 2024-03-22T19:12:34.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.094+0000 7fcabcc00700 1 -- 172.21.15.86:0/3820580893 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fcab0021580 con 0x7fcac0102f00 2024-03-22T19:12:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.094+0000 7fcada6e2b80 1 Processor -- start 2024-03-22T19:12:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 -- start start 2024-03-22T19:12:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47e9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa4740a10 con 0x55daa47e94a0 2024-03-22T19:12:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa473e3d0 con 0x55daa47e7ca0 2024-03-22T19:12:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa4739cc0 con 0x55daa47e8600 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcabcc00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcaa40a0a90 0x7fcaa40a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcac4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcabcc00700 1 -- 172.21.15.86:0/3820580893 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fcab00cd270 con 0x7fcac0102f00 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcabf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47e8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51482/0 (socket says 172.21.15.86:51482) 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.095+0000 7fcac4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47e80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34830/0 (socket says 172.21.15.86:34830) 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabfe00700 1 -- 172.21.15.86:0/1278044216 learned_addr learned my addr 172.21.15.86:0/1278044216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcac4c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcaa40a0a90 0x7fcaa40a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabf400700 1 -- 172.21.15.86:0/1278044216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 msgr2=0x55daa47e8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabf400700 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47e8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabf400700 1 -- 172.21.15.86:0/1278044216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 msgr2=0x55daa47e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabf400700 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabf400700 1 -- 172.21.15.86:0/1278044216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daa45bb430 con 0x55daa47e94a0 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcac4c00700 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcabf400700 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47e9920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcaac017bd0 tx=0x7fcaac017ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f08ac33e8a2d1b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.096+0000 7fcac4c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcaa40a0a90 0x7fcaa40a2f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009550 tx=0x7fcab800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.097+0000 7fcab5200700 1 -- 172.21.15.86:0/1278044216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac01c070 con 0x55daa47e94a0 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.097+0000 7fcab5200700 1 -- 172.21.15.86:0/1278044216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaac004030 con 0x55daa47e94a0 2024-03-22T19:12:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.097+0000 7fcab5200700 1 -- 172.21.15.86:0/1278044216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac004ca0 con 0x55daa47e94a0 2024-03-22T19:12:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.097+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1278044216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 msgr2=0x55daa47e9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.097+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47e9920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcaac017bd0 tx=0x7fcaac017ee0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1278044216 shutdown_connections 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47e9920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1278044216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1278044216 >> 172.21.15.86:0/1278044216 conn(0x55daa475d2a0 msgr2=0x55daa47e2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1278044216 shutdown_connections 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1278044216 wait complete. 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 -- start start 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.098+0000 7fcada6e2b80 1 Processor -- start 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e8600 0x55daa47f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa46d5e00 con 0x55daa47e94a0 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa46d5aa0 con 0x55daa47e8600 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa45bd720 con 0x55daa47e7ca0 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcada6e2b80 10 client.0 _ll_register_callbacks cb 0x55daa4541420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcac4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcac4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32968/0 (socket says 172.21.15.86:32968) 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcac4c00700 1 -- 172.21.15.86:0/2215567756 learned_addr learned my addr 172.21.15.86:0/2215567756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcabf400700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e8600 0x55daa47f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.099+0000 7fcabfe00700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcabf400700 1 -- 172.21.15.86:0/2215567756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e7ca0 msgr2=0x55daa47f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcabf400700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcabf400700 1 -- 172.21.15.86:0/2215567756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 msgr2=0x55daa47f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcabf400700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcabf400700 1 -- 172.21.15.86:0/2215567756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab8008a40 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcabf400700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e8600 0x55daa47f20b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009230 tx=0x7fcaac00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f744b82da4209712 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcaa9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcaa9600700 1 -- 172.21.15.86:0/2215567756 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac01c040 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcaa9600700 1 -- 172.21.15.86:0/2215567756 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fcaac005ce0 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daa47dab50 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcaa9600700 1 -- 172.21.15.86:0/2215567756 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaac00eeb0 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcada6e2b80 10 client.25811 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.100+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fcaac009c90 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcaa9600700 1 -- 172.21.15.86:0/2215567756 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaac038080 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcaa9600700 1 -- 172.21.15.86:0/2215567756 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcaac020940 con 0x55daa47e8600 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcada6e2b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcada6e2b80 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcada6e2b80 20 client.25811 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa47fd810 0x55daa47ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcada6e2b80 4 client.25811 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daa4732780 con 0x55daa47fd810 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.101+0000 7fcabfe00700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa47fd810 0x55daa47ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.102+0000 7fcaa9600700 10 client.25811 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.102+0000 7fcabfe00700 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa47fd810 0x55daa47ffc30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96247949109dcfa7 server_cookie=20dc778b50e7542f in_seq=0 out_seq=0 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.102+0000 7fcaa9600700 1 -- 172.21.15.86:0/2215567756 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55daa4732780 con 0x55daa47fd810 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.102+0000 7fcaa9600700 10 client.25811 handle_command_reply: tid=0 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.102+0000 7fcada6e2b80 1 client.25811 shutdown 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e8600 msgr2=0x55daa47f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e8600 0x55daa47f20b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009230 tx=0x7fcaac00ec50 comp rx=0 tx=0).stop 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 shutdown_connections 2024-03-22T19:12:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa47fd810 0x55daa47ffc30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcaa9600700 0 client.25811 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/2215567756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e8600 0x55daa47f20b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 >> 172.21.15.86:0/2215567756 conn(0x55daa475d2a0 msgr2=0x55daa47e2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 shutdown_connections 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 1 -- 172.21.15.86:0/2215567756 wait complete. 2024-03-22T19:12:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.103+0000 7fcada6e2b80 20 client.25811 trim_cache size 0 max 16384 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.120+0000 7fcada6e2b80 1 Processor -- start 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.120+0000 7fcada6e2b80 1 -- start start 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e94a0 0x55daa47f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa47e8440 con 0x55daa47e7ca0 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa47da690 con 0x55daa47e94a0 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa47f27f0 con 0x55daa47e8600 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcac4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e94a0 0x55daa47f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32980/0 (socket says 172.21.15.86:32980) 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 -- 172.21.15.86:0/3352766459 learned_addr learned my addr 172.21.15.86:0/3352766459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabf400700 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 -- 172.21.15.86:0/3352766459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 msgr2=0x55daa47f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 -- 172.21.15.86:0/3352766459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e94a0 msgr2=0x55daa47f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e94a0 0x55daa47f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 -- 172.21.15.86:0/3352766459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daa4567be0 con 0x55daa47e7ca0 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.121+0000 7fcabfe00700 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcab009a680 tx=0x7fcab00ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f274ed9beb674dee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcab5c00700 1 -- 172.21.15.86:0/3352766459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab00b3270 con 0x55daa47e7ca0 2024-03-22T19:12:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcab5c00700 1 -- 172.21.15.86:0/3352766459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab009a980 con 0x55daa47e7ca0 2024-03-22T19:12:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcab5c00700 1 -- 172.21.15.86:0/3352766459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0098e00 con 0x55daa47e7ca0 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 -- 172.21.15.86:0/3352766459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 msgr2=0x55daa47f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcab009a680 tx=0x7fcab00ba040 comp rx=0 tx=0).stop 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 -- 172.21.15.86:0/3352766459 shutdown_connections 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e94a0 0x55daa47f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e7ca0 0x55daa47f1b70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/3352766459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 -- 172.21.15.86:0/3352766459 >> 172.21.15.86:0/3352766459 conn(0x55daa475d2a0 msgr2=0x55daa47f0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.122+0000 7fcada6e2b80 1 -- 172.21.15.86:0/3352766459 shutdown_connections 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.123+0000 7fcada6e2b80 1 -- 172.21.15.86:0/3352766459 wait complete. 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.123+0000 7fcada6e2b80 1 -- start start 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.123+0000 7fcada6e2b80 1 Processor -- start 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa47f8230 con 0x55daa47e94a0 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa47f83a0 con 0x55daa47e7ca0 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa47f8560 con 0x55daa47e8600 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcada6e2b80 10 client.0 _ll_register_callbacks cb 0x7fcab8009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcac4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:34850/0 (socket says 172.21.15.86:34850) 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 -- 172.21.15.86:0/1973368148 learned_addr learned my addr 172.21.15.86:0/1973368148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabf400700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 -- 172.21.15.86:0/1973368148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 msgr2=0x55daa47f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 -- 172.21.15.86:0/1973368148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 msgr2=0x55daa47f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcabfe00700 1 -- 172.21.15.86:0/1973368148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a1310 con 0x55daa47e7ca0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.124+0000 7fcac4c00700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcabfe00700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47f6890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cad60 tx=0x7fcab009b1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e94bd0993bc3aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcaa9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcaa9600700 1 -- 172.21.15.86:0/1973368148 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0098e00 con 0x55daa47e7ca0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcaa9600700 1 -- 172.21.15.86:0/1973368148 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fcab00a2d70 con 0x55daa47e7ca0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaac004030 con 0x55daa47e7ca0 2024-03-22T19:12:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcaa9600700 1 -- 172.21.15.86:0/1973368148 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab009e7b0 con 0x55daa47e7ca0 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcada6e2b80 10 client.25817 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55daa46fff20 con 0x55daa47e7ca0 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.125+0000 7fcaa9600700 1 -- 172.21.15.86:0/1973368148 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaac004030 con 0x55daa47e7ca0 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcaa9600700 1 -- 172.21.15.86:0/1973368148 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcab00a28e0 con 0x55daa47e7ca0 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcada6e2b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcada6e2b80 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcada6e2b80 20 client.25817 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa4851830 0x55daa4853c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcada6e2b80 4 client.25817 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55daa4588d70 con 0x55daa4851830 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcabf400700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa4851830 0x55daa4853c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcabf400700 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa4851830 0x55daa4853c50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be5095ec402bdea server_cookie=491be6e56939806e in_seq=0 out_seq=0 2024-03-22T19:12:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.126+0000 7fcaa9600700 10 client.25817 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.127+0000 7fcaa9600700 1 -- 172.21.15.86:0/1973368148 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55daa4588d70 con 0x55daa4851830 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.127+0000 7fcaa9600700 10 client.25817 handle_command_reply: tid=0 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.127+0000 7fcada6e2b80 1 client.25817 shutdown 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.127+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 msgr2=0x55daa47f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.127+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47f6890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cad60 tx=0x7fcab009b1c0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 shutdown_connections 2024-03-22T19:12:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55daa4851830 0x55daa4853c50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55daa47e94a0 0x55daa47f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55daa47e7ca0 0x55daa47f6890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 --2- 172.21.15.86:0/1973368148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55daa47e8600 0x55daa47f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 >> 172.21.15.86:0/1973368148 conn(0x55daa475d2a0 msgr2=0x55daa47f1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcaa9600700 0 client.25817 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 shutdown_connections 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 1 -- 172.21.15.86:0/1973368148 wait complete. 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.128+0000 7fcada6e2b80 20 client.25817 trim_cache size 0 max 16384 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:12:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "07ae19be-1f4f-4218-b0ba-b22c29b38b55", 2024-03-22T19:12:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:12:34.134 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:12:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcaa40a0a90 msgr2=0x7fcaa40a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcaa40a0a90 0x7fcaa40a2f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009550 tx=0x7fcab800b040 comp rx=0 tx=0).stop 2024-03-22T19:12:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 msgr2=0x7fcac0198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 0x7fcac0198f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcab0009580 comp rx=0 tx=0).stop 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 shutdown_connections 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcaa40a0a90 0x7fcaa40a2f50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0104a90 0x7fcac01a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0102f00 0x7fcac0198f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 --2- 172.21.15.86:0/3820580893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac01040d0 0x7fcac0199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 >> 172.21.15.86:0/3820580893 conn(0x7fcac00fe4c0 msgr2=0x7fcac0107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 shutdown_connections 2024-03-22T19:12:34.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.129+0000 7fcac6c00700 1 -- 172.21.15.86:0/3820580893 wait complete. 2024-03-22T19:12:34.152 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:12:34.170 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: creating Makefile 2024-03-22T19:12:34.195 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: creating config.h 2024-03-22T19:12:34.231 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: executing depfiles commands 2024-03-22T19:12:34.261 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: executing default commands 2024-03-22T19:12:34.289 INFO:tasks.workunit.client.0.smithi086.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-03-22T19:12:34.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 -- 172.21.15.86:0/132933600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 msgr2=0x7fc310073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 --2- 172.21.15.86:0/132933600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc310073c90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009990 tx=0x7fc2fc009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 -- 172.21.15.86:0/132933600 shutdown_connections 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 --2- 172.21.15.86:0/132933600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 0x7fc31010f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 --2- 172.21.15.86:0/132933600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 0x7fc310073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 --2- 172.21.15.86:0/132933600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc310073c90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 -- 172.21.15.86:0/132933600 >> 172.21.15.86:0/132933600 conn(0x7fc3100fc2c0 msgr2=0x7fc3100fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 -- 172.21.15.86:0/132933600 shutdown_connections 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.374+0000 7fc315800700 1 -- 172.21.15.86:0/132933600 wait complete. 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 Processor -- start 2024-03-22T19:12:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 -- start start 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc31019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 0x7fc31019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 0x7fc3101a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101a4e60 con 0x7fc31010d320 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101a4fa0 con 0x7fc310073810 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.375+0000 7fc315800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101a5160 con 0x7fc310074df0 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 0x7fc3101a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc31019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 0x7fc3101a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33006/0 (socket says 172.21.15.86:33006) 2024-03-22T19:12:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30f400700 1 -- 172.21.15.86:0/4038383940 learned_addr learned my addr 172.21.15.86:0/4038383940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30e000700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 0x7fc31019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 -- 172.21.15.86:0/4038383940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 msgr2=0x7fc31019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 0x7fc31019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 -- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 msgr2=0x7fc3101a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 0x7fc3101a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 -- 172.21.15.86:0/4038383940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0095f0 con 0x7fc310073810 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30e000700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 0x7fc31019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.376+0000 7fc30ea00700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc31019d3d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc30000ea20 tx=0x7fc30000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.377+0000 7fc307400700 1 -- 172.21.15.86:0/4038383940 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000cbd0 con 0x7fc310073810 2024-03-22T19:12:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.377+0000 7fc307400700 1 -- 172.21.15.86:0/4038383940 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc30000ee60 con 0x7fc310073810 2024-03-22T19:12:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.377+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3101a5440 con 0x7fc310073810 2024-03-22T19:12:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.377+0000 7fc307400700 1 -- 172.21.15.86:0/4038383940 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300018680 con 0x7fc310073810 2024-03-22T19:12:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.377+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3101a59a0 con 0x7fc310073810 2024-03-22T19:12:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.380+0000 7fc307400700 1 -- 172.21.15.86:0/4038383940 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fc300010430 con 0x7fc310073810 2024-03-22T19:12:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.381+0000 7fc307400700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2f40a0b10 0x7fc2f40a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.381+0000 7fc307400700 1 -- 172.21.15.86:0/4038383940 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc300014070 con 0x7fc310073810 2024-03-22T19:12:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.381+0000 7fc3293cbb80 1 Processor -- start 2024-03-22T19:12:34.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.381+0000 7fc30e000700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2f40a0b10 0x7fc2f40a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 -- start start 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e71c20 0x557bd7e72040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 0x557bd7e72e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7dca990 con 0x557bd7e72580 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7dc8350 con 0x557bd7e73420 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7dc3c40 con 0x557bd7e71c20 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc30e000700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2f40a0b10 0x7fc2f40a2fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc30e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 0x557bd7e72e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e71c20 0x557bd7e72040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc30e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 0x557bd7e72e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33010/0 (socket says 172.21.15.86:33010) 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.382+0000 7fc30e000700 1 -- 172.21.15.86:0/1664189402 learned_addr learned my addr 172.21.15.86:0/1664189402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30f400700 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30ea00700 1 -- 172.21.15.86:0/1664189402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 msgr2=0x557bd7e738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30ea00700 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e738a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30ea00700 1 -- 172.21.15.86:0/1664189402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 msgr2=0x557bd7e72e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30ea00700 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 0x557bd7e72e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30ea00700 1 -- 172.21.15.86:0/1664189402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd7c453b0 con 0x557bd7e71c20 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30e000700 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 0x557bd7e72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc30ea00700 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e71c20 0x557bd7e72040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc300099470 tx=0x7fc3000995c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a4cc69fa3d70ff2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc2fbe00700 1 -- 172.21.15.86:0/1664189402 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000990d0 con 0x557bd7e71c20 2024-03-22T19:12:34.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc2fbe00700 1 -- 172.21.15.86:0/1664189402 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc300099e70 con 0x557bd7e71c20 2024-03-22T19:12:34.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.383+0000 7fc2fbe00700 1 -- 172.21.15.86:0/1664189402 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30009d120 con 0x557bd7e71c20 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1664189402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e71c20 msgr2=0x557bd7e72040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e71c20 0x557bd7e72040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc300099470 tx=0x7fc3000995c0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1664189402 shutdown_connections 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e71c20 0x557bd7e72040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1664189402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e72580 0x557bd7e72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1664189402 >> 172.21.15.86:0/1664189402 conn(0x557bd7de7220 msgr2=0x557bd7e6cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1664189402 shutdown_connections 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.384+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1664189402 wait complete. 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.385+0000 7fc3293cbb80 1 -- start start 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.385+0000 7fc3293cbb80 1 Processor -- start 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e83070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7d5fd80 con 0x557bd7e71c20 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7d5fa20 con 0x557bd7e73420 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7c476a0 con 0x557bd7e72580 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc3293cbb80 10 client.0 _ll_register_callbacks cb 0x557bd7bcb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc30e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc30f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e83070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc30e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e7c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51532/0 (socket says 172.21.15.86:51532) 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc30e000700 1 -- 172.21.15.86:0/1676237762 learned_addr learned my addr 172.21.15.86:0/1676237762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.386+0000 7fc30ea00700 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc30e000700 1 -- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 msgr2=0x557bd7e83070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc30e000700 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e83070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc30e000700 1 -- 172.21.15.86:0/1676237762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 msgr2=0x557bd7e7baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc30e000700 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e7baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc30e000700 1 -- 172.21.15.86:0/1676237762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3101100c0 con 0x557bd7e72580 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc30e000700 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e7c030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00b7b0 tx=0x7fc2fc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34698b6c28db4a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc2f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc2f8c00700 1 -- 172.21.15.86:0/1676237762 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc010020 con 0x557bd7e72580 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bd7e64ad0 con 0x557bd7e72580 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.387+0000 7fc2f8c00700 1 -- 172.21.15.86:0/1676237762 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc2fc010180 con 0x557bd7e72580 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc2f8c00700 1 -- 172.21.15.86:0/1676237762 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2fc024a30 con 0x557bd7e72580 2024-03-22T19:12:34.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 10 client.25819 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc3101a59a0 con 0x557bd7e72580 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc2f8c00700 1 -- 172.21.15.86:0/1676237762 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557bd7e64ad0 con 0x557bd7e72580 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc2f8c00700 1 -- 172.21.15.86:0/1676237762 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc2fc039070 con 0x557bd7e72580 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 10 client.25819 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 20 client.25819 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7e87920 0x557bd7e89d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 4 client.25819 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bd7dbc700 con 0x557bd7e87920 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.388+0000 7fc30ea00700 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7e87920 0x557bd7e89d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.389+0000 7fc2f8c00700 10 client.25819 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.389+0000 7fc30ea00700 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7e87920 0x557bd7e89d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b7e803a2e6a9b1 server_cookie=ce849a008d2629f8 in_seq=0 out_seq=0 2024-03-22T19:12:34.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.389+0000 7fc2f8c00700 1 -- 172.21.15.86:0/1676237762 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557bd7dbc700 con 0x557bd7e87920 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.389+0000 7fc2f8c00700 10 client.25819 handle_command_reply: tid=0 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.389+0000 7fc3293cbb80 1 client.25819 shutdown 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 msgr2=0x557bd7e7c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e7c030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00b7b0 tx=0x7fc2fc009ca0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 shutdown_connections 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7e87920 0x557bd7e89d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e83070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1676237762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e7c030 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc2f8c00700 0 client.25819 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 >> 172.21.15.86:0/1676237762 conn(0x557bd7de7220 msgr2=0x557bd7e6c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 shutdown_connections 2024-03-22T19:12:34.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1676237762 wait complete. 2024-03-22T19:12:34.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.390+0000 7fc3293cbb80 20 client.25819 trim_cache size 0 max 16384 2024-03-22T19:12:34.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 Processor -- start 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 -- start start 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e7a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7e67240 con 0x557bd7e71c20 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7e69630 con 0x557bd7e73420 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7e7cce0 con 0x557bd7e72580 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc30e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc30e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e73800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51546/0 (socket says 172.21.15.86:51546) 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e81020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33030/0 (socket says 172.21.15.86:33030) 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc30e000700 1 -- 172.21.15.86:0/1975326078 learned_addr learned my addr 172.21.15.86:0/1975326078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.407+0000 7fc30f400700 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e7a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc30e000700 1 -- 172.21.15.86:0/1975326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 msgr2=0x557bd7e7a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc30e000700 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e7a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc30e000700 1 -- 172.21.15.86:0/1975326078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 msgr2=0x557bd7e81020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc30e000700 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e81020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc30e000700 1 -- 172.21.15.86:0/1975326078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd7bf1b60 con 0x557bd7e72580 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc30e000700 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e73800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0098e0 tx=0x7fc2fc00fd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f37a42d5f1a1582 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc304c00700 1 -- 172.21.15.86:0/1975326078 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00fed0 con 0x557bd7e72580 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc304c00700 1 -- 172.21.15.86:0/1975326078 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2fc01e040 con 0x557bd7e72580 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.408+0000 7fc304c00700 1 -- 172.21.15.86:0/1975326078 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00f7c0 con 0x557bd7e72580 2024-03-22T19:12:34.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1975326078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 msgr2=0x557bd7e73800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e73800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0098e0 tx=0x7fc2fc00fd80 comp rx=0 tx=0).stop 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1975326078 shutdown_connections 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e73420 0x557bd7e7a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e81020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/1975326078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e72580 0x557bd7e73800 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1975326078 >> 172.21.15.86:0/1975326078 conn(0x557bd7de7220 msgr2=0x557bd7e79940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1975326078 shutdown_connections 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.409+0000 7fc3293cbb80 1 -- 172.21.15.86:0/1975326078 wait complete. 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 -- start start 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 Processor -- start 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e72580 0x557bd7e80e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e73420 0x557bd7e7ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7e7f600 con 0x557bd7e71c20 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7e7f770 con 0x557bd7e72580 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd7e7f930 con 0x557bd7e73420 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.410+0000 7fc3293cbb80 10 client.0 _ll_register_callbacks cb 0x7fc2fc0107c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e80910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e80910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33044/0 (socket says 172.21.15.86:33044) 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30ea00700 1 -- 172.21.15.86:0/110355147 learned_addr learned my addr 172.21.15.86:0/110355147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30f400700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e73420 0x557bd7e7ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e72580 0x557bd7e80e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 -- 172.21.15.86:0/110355147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e73420 msgr2=0x557bd7e7ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e73420 0x557bd7e7ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 -- 172.21.15.86:0/110355147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 msgr2=0x557bd7e80910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e80910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 -- 172.21.15.86:0/110355147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3101a59a0 con 0x557bd7e72580 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30ea00700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30f400700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e73420 0x557bd7e7ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc30e000700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e72580 0x557bd7e80e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc010480 tx=0x7fc2fc010090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173f34ab90395592 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.411+0000 7fc2f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc2f8c00700 1 -- 172.21.15.86:0/110355147 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc04a070 con 0x557bd7e72580 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc2f8c00700 1 -- 172.21.15.86:0/110355147 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc2fc024c40 con 0x557bd7e72580 2024-03-22T19:12:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc300099e70 con 0x557bd7e72580 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc2f8c00700 1 -- 172.21.15.86:0/110355147 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2fc024e00 con 0x557bd7e72580 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc3293cbb80 10 client.25835 fetch_fsmap learned FSMap version 27 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557bd7d89ea0 con 0x557bd7e72580 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc2f8c00700 1 -- 172.21.15.86:0/110355147 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2fc055020 con 0x557bd7e72580 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc2f8c00700 1 -- 172.21.15.86:0/110355147 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc2fc04d020 con 0x557bd7e72580 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc3293cbb80 10 client.25835 fetch_fsmap finished waiting for FSMap version 27 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.412+0000 7fc3293cbb80 10 client.25835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc3293cbb80 20 client.25835 populate_metadata read hostname 'smithi086' 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7eda2d0 0x557bd7edc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:12:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc3293cbb80 4 client.25835 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557bd7c12cf0 con 0x557bd7eda2d0 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc30ea00700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7eda2d0 0x557bd7edc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc2f8c00700 10 client.25835 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc30ea00700 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7eda2d0 0x557bd7edc6f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c71e83fe6c2a server_cookie=ebdc8c9c09b83d2a in_seq=0 out_seq=0 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc2f8c00700 1 -- 172.21.15.86:0/110355147 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557bd7c12cf0 con 0x557bd7eda2d0 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.413+0000 7fc2f8c00700 10 client.25835 handle_command_reply: tid=0 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 client.25835 shutdown 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e72580 msgr2=0x557bd7e80e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e72580 0x557bd7e80e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc010480 tx=0x7fc2fc010090 comp rx=0 tx=0).stop 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 shutdown_connections 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557bd7eda2d0 0x557bd7edc6f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bd7e73420 0x557bd7e7ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc2f8c00700 0 client.25835 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:12:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bd7e71c20 0x557bd7e80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 --2- 172.21.15.86:0/110355147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557bd7e72580 0x557bd7e80e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 >> 172.21.15.86:0/110355147 conn(0x557bd7de7220 msgr2=0x557bd7e79870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 shutdown_connections 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 1 -- 172.21.15.86:0/110355147 wait complete. 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.414+0000 7fc3293cbb80 20 client.25835 trim_cache size 0 max 16384 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stdout: "07ae19be-1f4f-4218-b0ba-b22c29b38b55": { 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:12:34.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "07ae19be-1f4f-4218-b0ba-b22c29b38b55", 2024-03-22T19:12:34.421 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:12:34.421 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:12:34.421 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:12:34.421 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:12:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2f40a0b10 msgr2=0x7fc2f40a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2f40a0b10 0x7fc2f40a2fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc005fb0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 msgr2=0x7fc31019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc31019d3d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc30000ea20 tx=0x7fc30000ed30 comp rx=0 tx=0).stop 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 shutdown_connections 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc2f40a0b10 0x7fc2f40a2fd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31010d320 0x7fc3101a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc310073810 0x7fc31019d3d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 --2- 172.21.15.86:0/4038383940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc310074df0 0x7fc31019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 >> 172.21.15.86:0/4038383940 conn(0x7fc3100fc2c0 msgr2=0x7fc3101006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 shutdown_connections 2024-03-22T19:12:34.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:12:34.415+0000 7fc315800700 1 -- 172.21.15.86:0/4038383940 wait complete. 2024-03-22T19:12:34.435 INFO:tasks.cephfs.filesystem:scrub status for tag:07ae19be-1f4f-4218-b0ba-b22c29b38b55 - {'path': '/', 'tag': '07ae19be-1f4f-4218-b0ba-b22c29b38b55', 'options': 'recursive,force'} 2024-03-22T19:12:34.494 INFO:tasks.workunit.client.0.smithi086.stdout:rm -f stamp-h1 2024-03-22T19:12:34.497 INFO:tasks.workunit.client.0.smithi086.stdout:touch config.h.in 2024-03-22T19:12:34.500 INFO:tasks.workunit.client.0.smithi086.stdout:cd . && /bin/sh ./config.status config.h 2024-03-22T19:12:34.577 INFO:tasks.workunit.client.0.smithi086.stdout:config.status: creating config.h 2024-03-22T19:12:34.612 INFO:tasks.workunit.client.0.smithi086.stdout:make all-am 2024-03-22T19:12:34.620 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-03-22T19:12:34.620 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-03-22T19:12:34.812 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-03-22T19:12:34.817 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-03-22T19:12:34.907 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-03-22T19:12:34.912 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-03-22T19:12:35.028 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-03-22T19:12:35.034 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-03-22T19:12:35.148 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-03-22T19:12:35.153 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-03-22T19:12:35.296 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-03-22T19:12:35.302 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-03-22T19:12:35.370 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-03-22T19:12:35.376 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-03-22T19:12:35.435 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-03-22T19:12:35.440 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-03-22T19:12:35.504 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-03-22T19:12:35.509 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-03-22T19:12:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:35 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:12:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:35 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:35 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:12:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:35 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:35 smithi167 ceph-mon[47728]: pgmap v334: 97 pgs: 97 active+clean; 5.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 453 KiB/s wr, 55 op/s 2024-03-22T19:12:35.690 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-03-22T19:12:35.695 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-03-22T19:12:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:35 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:12:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:35 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:35.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:35 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:12:35.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:35 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:35.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:35 smithi131 ceph-mon[78559]: pgmap v334: 97 pgs: 97 active+clean; 5.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 453 KiB/s wr, 55 op/s 2024-03-22T19:12:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:35 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:12:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:35 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:35 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:12:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:35 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:12:35.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:35 smithi086 ceph-mon[38032]: pgmap v334: 97 pgs: 97 active+clean; 5.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 453 KiB/s wr, 55 op/s 2024-03-22T19:12:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-03-22T19:12:35.834 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-03-22T19:12:35.994 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-03-22T19:12:36.000 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-03-22T19:12:36.085 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-03-22T19:12:36.090 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-03-22T19:12:36.179 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-03-22T19:12:36.184 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-03-22T19:12:36.308 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-03-22T19:12:36.314 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr:In file included from parser.h:22, 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr: from parser.c:25: 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c: In function ‘init_filesys’: 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.352 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.353 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.354 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2024-03-22T19:12:36.355 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2024-03-22T19:12:36.356 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c: In function ‘init_tg_stats’: 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.357 INFO:tasks.workunit.client.0.smithi086.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.358 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.359 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: ^ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: container_of(ptr, type, member) 2024-03-22T19:12:36.360 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~ 2024-03-22T19:12:36.361 INFO:tasks.workunit.client.0.smithi086.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-03-22T19:12:36.361 INFO:tasks.workunit.client.0.smithi086.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-03-22T19:12:36.361 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~ 2024-03-22T19:12:36.361 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2024-03-22T19:12:36.361 INFO:tasks.workunit.client.0.smithi086.stderr: list_for_each_entry(tmp_list, 2024-03-22T19:12:36.361 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c: In function ‘set_weight’: 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:623:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr: strncpy (op, tmp_config->name, len - 7); 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:620:10: note: length computed here 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr: len = strlen(tmp_config->name); 2024-03-22T19:12:36.467 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.471 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c: In function ‘get_weight_total’: 2024-03-22T19:12:36.472 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:645:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-03-22T19:12:36.472 INFO:tasks.workunit.client.0.smithi086.stderr: strncpy (op, tmp_config->name, len - 7); 2024-03-22T19:12:36.472 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.472 INFO:tasks.workunit.client.0.smithi086.stderr:parser.c:642:10: note: length computed here 2024-03-22T19:12:36.472 INFO:tasks.workunit.client.0.smithi086.stderr: len = strlen(tmp_config->name); 2024-03-22T19:12:36.472 INFO:tasks.workunit.client.0.smithi086.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-22T19:12:36.610 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-03-22T19:12:36.616 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-03-22T19:12:36.691 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-03-22T19:12:36.697 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-03-22T19:12:36.804 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-03-22T19:12:36.809 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-03-22T19:12:36.923 INFO:tasks.workunit.client.0.smithi086.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-03-22T19:12:36.930 INFO:tasks.workunit.client.0.smithi086.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-03-22T19:12:36.978 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-03-22T19:12:36.984 INFO:tasks.workunit.client.0.smithi086.stdout:FFSB version 6.0-RC2 started 2024-03-22T19:12:36.984 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout:benchmark time = 300 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout:ThreadGroup 0 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout:================ 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout: num_threads = 32 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout: read_random = off 2024-03-22T19:12:36.990 INFO:tasks.workunit.client.0.smithi086.stdout: read_size = 0 (0B) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: read_blocksize = 0 (0B) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: read_skip = off 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: read_skipsize = 0 (0B) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: write_random = on 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: write_size = 5242880 (5MB) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: fsync_file = 0 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: write_blocksize = 4096 (4KB) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: wait time = 0 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: op weights 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: read = 0 (0.00%) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: readall = 0 (0.00%) 2024-03-22T19:12:36.991 INFO:tasks.workunit.client.0.smithi086.stdout: write = 1 (100.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: create = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: append = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: delete = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: metaop = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: createdir = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: stat = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: writeall = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: writeall_fsync = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: open_close = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: write_fsync = 0 (0.00%) 2024-03-22T19:12:36.992 INFO:tasks.workunit.client.0.smithi086.stdout: create_fsync = 0 (0.00%) 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: append_fsync = 0 (0.00%) 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout:FileSystem . 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout:========== 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: num_dirs = 0 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: starting files = 128 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: min file size = 104857600 (100MB) 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: max file size = 104857600 (100MB) 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: directio = off 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: alignedio = on 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: bufferedio = off 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.993 INFO:tasks.workunit.client.0.smithi086.stdout: aging is off 2024-03-22T19:12:36.994 INFO:tasks.workunit.client.0.smithi086.stdout: current utilization = 0.00% 2024-03-22T19:12:36.994 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:12:36.994 INFO:tasks.workunit.client.0.smithi086.stdout:checking existing fs: . 2024-03-22T19:12:36.994 INFO:tasks.workunit.client.0.smithi086.stdout:recreating new fileset 2024-03-22T19:12:36.995 INFO:tasks.workunit.client.0.smithi086.stderr:opendir: No such file or directory 2024-03-22T19:12:37.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:37 smithi131 ceph-mon[78559]: pgmap v335: 97 pgs: 97 active+clean; 5.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 349 KiB/s wr, 49 op/s 2024-03-22T19:12:37.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:37 smithi086 ceph-mon[38032]: pgmap v335: 97 pgs: 97 active+clean; 5.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 349 KiB/s wr, 49 op/s 2024-03-22T19:12:37.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:37 smithi167 ceph-mon[47728]: pgmap v335: 97 pgs: 97 active+clean; 5.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 349 KiB/s wr, 49 op/s 2024-03-22T19:12:38.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:38 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:12:38.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:38 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:12:38.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:38 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:12:39.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:39 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:12:39.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:39 smithi131 ceph-mon[78559]: pgmap v336: 97 pgs: 97 active+clean; 6.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 446 KiB/s wr, 56 op/s 2024-03-22T19:12:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:39 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:12:39.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:39 smithi086 ceph-mon[38032]: pgmap v336: 97 pgs: 97 active+clean; 6.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 446 KiB/s wr, 56 op/s 2024-03-22T19:12:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:39 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:12:39.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:39 smithi167 ceph-mon[47728]: pgmap v336: 97 pgs: 97 active+clean; 6.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 446 KiB/s wr, 56 op/s 2024-03-22T19:12:41.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:41 smithi167 ceph-mon[47728]: pgmap v337: 97 pgs: 97 active+clean; 6.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 324 KiB/s wr, 55 op/s 2024-03-22T19:12:41.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:41 smithi131 ceph-mon[78559]: pgmap v337: 97 pgs: 97 active+clean; 6.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 324 KiB/s wr, 55 op/s 2024-03-22T19:12:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:41 smithi086 ceph-mon[38032]: pgmap v337: 97 pgs: 97 active+clean; 6.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 324 KiB/s wr, 55 op/s 2024-03-22T19:12:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:43 smithi167 ceph-mon[47728]: pgmap v338: 97 pgs: 97 active+clean; 8.8 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 533 KiB/s wr, 88 op/s 2024-03-22T19:12:43.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:43 smithi131 ceph-mon[78559]: pgmap v338: 97 pgs: 97 active+clean; 8.8 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 533 KiB/s wr, 88 op/s 2024-03-22T19:12:43.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:43 smithi086 ceph-mon[38032]: pgmap v338: 97 pgs: 97 active+clean; 8.8 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 533 KiB/s wr, 88 op/s 2024-03-22T19:12:45.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:45 smithi167 ceph-mon[47728]: pgmap v339: 97 pgs: 97 active+clean; 29 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 2.2 MiB/s wr, 75 op/s 2024-03-22T19:12:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:45 smithi131 ceph-mon[78559]: pgmap v339: 97 pgs: 97 active+clean; 29 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 2.2 MiB/s wr, 75 op/s 2024-03-22T19:12:45.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:45 smithi086 ceph-mon[38032]: pgmap v339: 97 pgs: 97 active+clean; 29 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 2.2 MiB/s wr, 75 op/s 2024-03-22T19:12:47.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:47 smithi131 ceph-mon[78559]: pgmap v340: 97 pgs: 97 active+clean; 253 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 21 MiB/s wr, 74 op/s 2024-03-22T19:12:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:47 smithi086 ceph-mon[38032]: pgmap v340: 97 pgs: 97 active+clean; 253 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 21 MiB/s wr, 74 op/s 2024-03-22T19:12:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:47 smithi167 ceph-mon[47728]: pgmap v340: 97 pgs: 97 active+clean; 253 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 21 MiB/s wr, 74 op/s 2024-03-22T19:12:49.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:49 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:12:49.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:49 smithi131 ceph-mon[78559]: pgmap v341: 97 pgs: 97 active+clean; 558 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 46 MiB/s wr, 92 op/s 2024-03-22T19:12:49.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:49 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:12:49.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:49 smithi086 ceph-mon[38032]: pgmap v341: 97 pgs: 97 active+clean; 558 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 46 MiB/s wr, 92 op/s 2024-03-22T19:12:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:49 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:12:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:49 smithi167 ceph-mon[47728]: pgmap v341: 97 pgs: 97 active+clean; 558 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 46 MiB/s wr, 92 op/s 2024-03-22T19:12:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:51 smithi167 ceph-mon[47728]: pgmap v342: 97 pgs: 97 active+clean; 978 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 81 MiB/s wr, 97 op/s 2024-03-22T19:12:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:51 smithi131 ceph-mon[78559]: pgmap v342: 97 pgs: 97 active+clean; 978 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 81 MiB/s wr, 97 op/s 2024-03-22T19:12:51.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:51 smithi086 ceph-mon[38032]: pgmap v342: 97 pgs: 97 active+clean; 978 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 81 MiB/s wr, 97 op/s 2024-03-22T19:12:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:53 smithi167 ceph-mon[47728]: pgmap v343: 97 pgs: 97 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 165 MiB/s wr, 112 op/s 2024-03-22T19:12:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:53 smithi131 ceph-mon[78559]: pgmap v343: 97 pgs: 97 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 165 MiB/s wr, 112 op/s 2024-03-22T19:12:53.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:53 smithi086 ceph-mon[38032]: pgmap v343: 97 pgs: 97 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 165 MiB/s wr, 112 op/s 2024-03-22T19:12:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:55 smithi167 ceph-mon[47728]: pgmap v344: 97 pgs: 97 active+clean; 2.2 GiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 183 MiB/s wr, 76 op/s 2024-03-22T19:12:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:55 smithi131 ceph-mon[78559]: pgmap v344: 97 pgs: 97 active+clean; 2.2 GiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 183 MiB/s wr, 76 op/s 2024-03-22T19:12:55.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:55 smithi086 ceph-mon[38032]: pgmap v344: 97 pgs: 97 active+clean; 2.2 GiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 183 MiB/s wr, 76 op/s 2024-03-22T19:12:57.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:57 smithi131 ceph-mon[78559]: pgmap v345: 97 pgs: 97 active+clean; 2.8 GiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 233 MiB/s wr, 97 op/s 2024-03-22T19:12:57.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:57 smithi086 ceph-mon[38032]: pgmap v345: 97 pgs: 97 active+clean; 2.8 GiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 233 MiB/s wr, 97 op/s 2024-03-22T19:12:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:57 smithi167 ceph-mon[47728]: pgmap v345: 97 pgs: 97 active+clean; 2.8 GiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 233 MiB/s wr, 97 op/s 2024-03-22T19:12:59.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:12:59 smithi086 ceph-mon[38032]: pgmap v346: 97 pgs: 97 active+clean; 3.2 GiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 MiB/s wr, 103 op/s 2024-03-22T19:12:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:12:59 smithi167 ceph-mon[47728]: pgmap v346: 97 pgs: 97 active+clean; 3.2 GiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 MiB/s wr, 103 op/s 2024-03-22T19:12:59.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:12:59 smithi131 ceph-mon[78559]: pgmap v346: 97 pgs: 97 active+clean; 3.2 GiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 MiB/s wr, 103 op/s 2024-03-22T19:13:01.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:01 smithi086 ceph-mon[38032]: pgmap v347: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 260 MiB/s wr, 101 op/s 2024-03-22T19:13:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:01 smithi167 ceph-mon[47728]: pgmap v347: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 260 MiB/s wr, 101 op/s 2024-03-22T19:13:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:01 smithi131 ceph-mon[78559]: pgmap v347: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 260 MiB/s wr, 101 op/s 2024-03-22T19:13:03.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:03 smithi086 ceph-mon[38032]: pgmap v348: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 305 MiB/s wr, 109 op/s 2024-03-22T19:13:03.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:03 smithi167 ceph-mon[47728]: pgmap v348: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 305 MiB/s wr, 109 op/s 2024-03-22T19:13:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:03 smithi131 ceph-mon[78559]: pgmap v348: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 305 MiB/s wr, 109 op/s 2024-03-22T19:13:04.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:13:04.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 -- 172.21.15.86:0/1749348215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69981028d0 msgr2=0x7f6998109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 --2- 172.21.15.86:0/1749348215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69981028d0 0x7f6998109180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6990009f90 tx=0x7f699000b330 comp rx=0 tx=0).stop 2024-03-22T19:13:04.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 -- 172.21.15.86:0/1749348215 shutdown_connections 2024-03-22T19:13:04.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 --2- 172.21.15.86:0/1749348215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69981028d0 0x7f6998109180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 --2- 172.21.15.86:0/1749348215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998100d40 0x7f6998101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 --2- 172.21.15.86:0/1749348215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6998101f30 0x7f6998102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.765+0000 7f699ee00700 1 -- 172.21.15.86:0/1749348215 >> 172.21.15.86:0/1749348215 conn(0x7f69980fc2b0 msgr2=0x7f69980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:04.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.766+0000 7f699ee00700 1 -- 172.21.15.86:0/1749348215 shutdown_connections 2024-03-22T19:13:04.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.766+0000 7f699ee00700 1 -- 172.21.15.86:0/1749348215 wait complete. 2024-03-22T19:13:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 Processor -- start 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 -- start start 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6998100d40 0x7f699819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 0x7f699819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69981028d0 0x7f69981a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981a4ee0 con 0x7f6998101f30 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981a5050 con 0x7f69981028d0 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f699ee00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981a5210 con 0x7f6998100d40 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f6997400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 0x7f699819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f6997e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6998100d40 0x7f699819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.767+0000 7f6997400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 0x7f699819da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38366/0 (socket says 172.21.15.86:38366) 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 -- 172.21.15.86:0/4116768748 learned_addr learned my addr 172.21.15.86:0/4116768748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f699ce00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69981028d0 0x7f69981a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 -- 172.21.15.86:0/4116768748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6998100d40 msgr2=0x7f699819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6998100d40 0x7f699819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 -- 172.21.15.86:0/4116768748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69981028d0 msgr2=0x7f69981a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69981028d0 0x7f69981a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 -- 172.21.15.86:0/4116768748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840095f0 con 0x7f6998101f30 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f699ce00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69981028d0 0x7f69981a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6997400700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 0x7f699819da00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f698400ec10 tx=0x7f698400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.768+0000 7f6994c00700 1 -- 172.21.15.86:0/4116768748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400cc90 con 0x7f6998101f30 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.769+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6990009c40 con 0x7f6998101f30 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.769+0000 7f6994c00700 1 -- 172.21.15.86:0/4116768748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69840044e0 con 0x7f6998101f30 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.769+0000 7f6994c00700 1 -- 172.21.15.86:0/4116768748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984010430 con 0x7f6998101f30 2024-03-22T19:13:04.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.769+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69981a58c0 con 0x7f6998101f30 2024-03-22T19:13:04.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.773+0000 7f6994c00700 1 -- 172.21.15.86:0/4116768748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6984003fb0 con 0x7f6998101f30 2024-03-22T19:13:04.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.775+0000 7f6994c00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f697c0a0b10 0x7f697c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.776+0000 7f6994c00700 1 -- 172.21.15.86:0/4116768748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6984014070 con 0x7f6998101f30 2024-03-22T19:13:04.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.776+0000 7f6997e00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f697c0a0b10 0x7f697c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.777+0000 7f69b2a1db80 1 Processor -- start 2024-03-22T19:13:04.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.777+0000 7f69b2a1db80 1 -- start start 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3617f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f6997e00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f697c0a0b10 0x7f697c0a2fd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f699819e870 tx=0x7f6988009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a3617fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a360d7990 con 0x558a3617ec20 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a360d5350 con 0x558a3617f580 2024-03-22T19:13:04.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a360d0c40 con 0x558a36180420 2024-03-22T19:13:04.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f6997e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3617f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f6997400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a3617fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f6997e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3617f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38376/0 (socket says 172.21.15.86:38376) 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.778+0000 7f6997e00700 1 -- 172.21.15.86:0/2253221519 learned_addr learned my addr 172.21.15.86:0/2253221519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f699ce00700 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f6997e00700 1 -- 172.21.15.86:0/2253221519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 msgr2=0x558a361808a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f6997e00700 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361808a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f6997e00700 1 -- 172.21.15.86:0/2253221519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 msgr2=0x558a3617fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f6997e00700 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a3617fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f6997e00700 1 -- 172.21.15.86:0/2253221519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a35f523b0 con 0x558a3617ec20 2024-03-22T19:13:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.779+0000 7f6997e00700 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3617f040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6988011d30 tx=0x7f6988014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bc7c1972005693 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.780+0000 7f698d600700 1 -- 172.21.15.86:0/2253221519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698800eb20 con 0x558a3617ec20 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.780+0000 7f698d600700 1 -- 172.21.15.86:0/2253221519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6988021070 con 0x558a3617ec20 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.780+0000 7f698d600700 1 -- 172.21.15.86:0/2253221519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698801cea0 con 0x558a3617ec20 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2253221519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 msgr2=0x558a3617f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3617f040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6988011d30 tx=0x7f6988014000 comp rx=0 tx=0).stop 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2253221519 shutdown_connections 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361808a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3617f040 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2253221519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a3617fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2253221519 >> 172.21.15.86:0/2253221519 conn(0x558a360f4220 msgr2=0x558a36179bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2253221519 shutdown_connections 2024-03-22T19:13:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.781+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2253221519 wait complete. 2024-03-22T19:13:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.782+0000 7f69b2a1db80 1 -- start start 2024-03-22T19:13:04.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.782+0000 7f69b2a1db80 1 Processor -- start 2024-03-22T19:13:04.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a36188b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36189080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a361900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a3606cd80 con 0x558a3617ec20 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a3606ca20 con 0x558a36180420 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a35f546a0 con 0x558a3617f580 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.783+0000 7f69b2a1db80 10 client.0 _ll_register_callbacks cb 0x558a35ed83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a36188b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36189080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a36188b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38386/0 (socket says 172.21.15.86:38386) 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997e00700 1 -- 172.21.15.86:0/2914092036 learned_addr learned my addr 172.21.15.86:0/2914092036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f699ce00700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a361900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 -- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 msgr2=0x558a361900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a361900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 -- 172.21.15.86:0/2914092036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 msgr2=0x558a36188b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a36188b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 -- 172.21.15.86:0/2914092036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6988017070 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f699ce00700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a361900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6997400700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36189080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6984096080 tx=0x7f6984099560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aac7f64fb0da9dd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.784+0000 7f6982000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f6982000700 1 -- 172.21.15.86:0/2914092036 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984099b80 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f6982000700 1 -- 172.21.15.86:0/2914092036 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f698409d9b0 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a36171ad0 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f6982000700 1 -- 172.21.15.86:0/2914092036 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69840a2000 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 10 client.25837 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558a35efeb60 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f6982000700 1 -- 172.21.15.86:0/2914092036 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a36171ad0 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f6982000700 1 -- 172.21.15.86:0/2914092036 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6984099100 con 0x558a3617f580 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 10 client.25837 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 20 client.25837 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.785+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361949d0 0x558a36196df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.786+0000 7f69b2a1db80 4 client.25837 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.786+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a360c9700 con 0x558a361949d0 2024-03-22T19:13:04.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.786+0000 7f6997e00700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361949d0 0x558a36196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.786+0000 7f6997e00700 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361949d0 0x558a36196df0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e683a32ffe927a76 server_cookie=261a14305bdf1041 in_seq=0 out_seq=0 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.786+0000 7f6982000700 10 client.25837 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f6982000700 1 -- 172.21.15.86:0/2914092036 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558a360c9700 con 0x558a361949d0 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f6982000700 10 client.25837 handle_command_reply: tid=0 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 client.25837 shutdown 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 msgr2=0x558a36189080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36189080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6984096080 tx=0x7f6984099560 comp rx=0 tx=0).stop 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 shutdown_connections 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361949d0 0x558a36196df0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a361900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a36188b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.787+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/2914092036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36189080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.788+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 >> 172.21.15.86:0/2914092036 conn(0x558a360f4220 msgr2=0x558a361791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.788+0000 7f6982000700 0 client.25837 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.788+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 shutdown_connections 2024-03-22T19:13:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.788+0000 7f69b2a1db80 1 -- 172.21.15.86:0/2914092036 wait complete. 2024-03-22T19:13:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.788+0000 7f69b2a1db80 20 client.25837 trim_cache size 0 max 16384 2024-03-22T19:13:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.806+0000 7f69b2a1db80 1 Processor -- start 2024-03-22T19:13:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.806+0000 7f69b2a1db80 1 -- start start 2024-03-22T19:13:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3618e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36180800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a36188b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a36144970 con 0x558a3617ec20 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a36176610 con 0x558a36180420 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a36191dd0 con 0x558a3617f580 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36180800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36180800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47032/0 (socket says 172.21.15.86:47032) 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 -- 172.21.15.86:0/1075769916 learned_addr learned my addr 172.21.15.86:0/1075769916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997e00700 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3618e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 -- 172.21.15.86:0/1075769916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 msgr2=0x558a36188b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a36188b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 -- 172.21.15.86:0/1075769916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 msgr2=0x558a3618e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3618e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.807+0000 7f6997400700 1 -- 172.21.15.86:0/1075769916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a36096ea0 con 0x558a3617f580 2024-03-22T19:13:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.808+0000 7f6997400700 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36180800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f698400c5b0 tx=0x7f698400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41556f631ab3253b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.808+0000 7f698e000700 1 -- 172.21.15.86:0/1075769916 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69840a1e00 con 0x558a3617f580 2024-03-22T19:13:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.808+0000 7f698e000700 1 -- 172.21.15.86:0/1075769916 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f698409d9b0 con 0x558a3617f580 2024-03-22T19:13:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.808+0000 7f698e000700 1 -- 172.21.15.86:0/1075769916 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984099070 con 0x558a3617f580 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 -- 172.21.15.86:0/1075769916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 msgr2=0x558a36180800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36180800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f698400c5b0 tx=0x7f698400ef90 comp rx=0 tx=0).stop 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 -- 172.21.15.86:0/1075769916 shutdown_connections 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a36180420 0x558a36188b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a3618e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/1075769916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a3617f580 0x558a36180800 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 -- 172.21.15.86:0/1075769916 >> 172.21.15.86:0/1075769916 conn(0x558a360f4220 msgr2=0x558a36175a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 -- 172.21.15.86:0/1075769916 shutdown_connections 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.809+0000 7f69b2a1db80 1 -- 172.21.15.86:0/1075769916 wait complete. 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.810+0000 7f69b2a1db80 1 -- start start 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.810+0000 7f69b2a1db80 1 Processor -- start 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a361de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a361dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361e5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a361e5b50 con 0x558a3617ec20 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a361e5cc0 con 0x558a3617f580 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a361e5e80 con 0x558a36180420 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f69b2a1db80 10 client.0 _ll_register_callbacks cb 0x7f6984099970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f699ce00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f699ce00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47036/0 (socket says 172.21.15.86:47036) 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.811+0000 7f699ce00700 1 -- 172.21.15.86:0/3072728295 learned_addr learned my addr 172.21.15.86:0/3072728295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f699ce00700 1 -- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 msgr2=0x558a361dec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f699ce00700 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a361dec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f699ce00700 1 -- 172.21.15.86:0/3072728295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 msgr2=0x558a361de6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f699ce00700 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a361de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f699ce00700 1 -- 172.21.15.86:0/3072728295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a35efeb60 con 0x558a36180420 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f699ce00700 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361e5610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6990009f90 tx=0x7f6990003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79864785d76c03a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f6982000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f6982000700 1 -- 172.21.15.86:0/3072728295 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699000e040 con 0x558a36180420 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.812+0000 7f6982000700 1 -- 172.21.15.86:0/3072728295 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f699000bbf0 con 0x558a36180420 2024-03-22T19:13:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6988021070 con 0x558a36180420 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f6982000700 1 -- 172.21.15.86:0/3072728295 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6990009410 con 0x558a36180420 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 10 client.25843 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558a35f56cd0 con 0x558a36180420 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f6982000700 1 -- 172.21.15.86:0/3072728295 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6988021070 con 0x558a36180420 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f6982000700 1 -- 172.21.15.86:0/3072728295 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f699000b050 con 0x558a36180420 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 10 client.25843 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 20 client.25843 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361e7940 0x558a361e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 4 client.25843 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.813+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558a35f1fcf0 con 0x558a361e7940 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.814+0000 7f6997e00700 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361e7940 0x558a361e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.814+0000 7f6982000700 10 client.25843 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.814+0000 7f6997e00700 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361e7940 0x558a361e9d60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d691173844d60c6a server_cookie=f35aef3060b2c147 in_seq=0 out_seq=0 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.814+0000 7f6982000700 1 -- 172.21.15.86:0/3072728295 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558a35f1fcf0 con 0x558a361e7940 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.814+0000 7f6982000700 10 client.25843 handle_command_reply: tid=0 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.814+0000 7f69b2a1db80 1 client.25843 shutdown 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 msgr2=0x558a361e5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361e5610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6990009f90 tx=0x7f6990003cc0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 shutdown_connections 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558a361e7940 0x558a361e9d60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a36180420 0x558a361e5610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a3617ec20 0x558a361de6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 --2- 172.21.15.86:0/3072728295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558a3617f580 0x558a361dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f6982000700 0 client.25843 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 >> 172.21.15.86:0/3072728295 conn(0x558a360f4220 msgr2=0x558a361833e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 shutdown_connections 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 1 -- 172.21.15.86:0/3072728295 wait complete. 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.815+0000 7f69b2a1db80 20 client.25843 trim_cache size 0 max 16384 2024-03-22T19:13:04.824 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:13:04.825 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:13:04.825 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:13:04.825 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f697c0a0b10 msgr2=0x7f697c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f697c0a0b10 0x7f697c0a2fd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f699819e870 tx=0x7f6988009260 comp rx=0 tx=0).stop 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 msgr2=0x7f699819da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 0x7f699819da00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f698400ec10 tx=0x7f698400ef20 comp rx=0 tx=0).stop 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 shutdown_connections 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f697c0a0b10 0x7f697c0a2fd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69981028d0 0x7f69981a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6998100d40 0x7f699819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 --2- 172.21.15.86:0/4116768748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6998101f30 0x7f699819da00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 >> 172.21.15.86:0/4116768748 conn(0x7f69980fc2b0 msgr2=0x7f69980fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 shutdown_connections 2024-03-22T19:13:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:04.816+0000 7f699ee00700 1 -- 172.21.15.86:0/4116768748 wait complete. 2024-03-22T19:13:04.835 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:13:04.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 -- 172.21.15.86:0/3043035712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8104980 msgr2=0x7fb6e810b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 --2- 172.21.15.86:0/3043035712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8104980 0x7fb6e810b2a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000b0a0 tx=0x7fb6e000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 -- 172.21.15.86:0/3043035712 shutdown_connections 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 --2- 172.21.15.86:0/3043035712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8104980 0x7fb6e810b2a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 --2- 172.21.15.86:0/3043035712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e8102e70 0x7fb6e8103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 --2- 172.21.15.86:0/3043035712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8103fc0 0x7fb6e8104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.179+0000 7fb6ed200700 1 -- 172.21.15.86:0/3043035712 >> 172.21.15.86:0/3043035712 conn(0x7fb6e80fe420 msgr2=0x7fb6e8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.180+0000 7fb6ed200700 1 -- 172.21.15.86:0/3043035712 shutdown_connections 2024-03-22T19:13:05.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.180+0000 7fb6ed200700 1 -- 172.21.15.86:0/3043035712 wait complete. 2024-03-22T19:13:05.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.181+0000 7fb6ed200700 1 Processor -- start 2024-03-22T19:13:05.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.181+0000 7fb6ed200700 1 -- start start 2024-03-22T19:13:05.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6ed200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102e70 0x7fb6e806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6ed200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e8103fc0 0x7fb6e806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6ed200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8104980 0x7fb6e81a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6ed200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80702f0 con 0x7fb6e8102e70 2024-03-22T19:13:05.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6ed200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8070430 con 0x7fb6e8103fc0 2024-03-22T19:13:05.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6ed200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e81a9590 con 0x7fb6e8104980 2024-03-22T19:13:05.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6e6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8104980 0x7fb6e81a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.182+0000 7fb6e6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8104980 0x7fb6e81a8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47062/0 (socket says 172.21.15.86:47062) 2024-03-22T19:13:05.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e6800700 1 -- 172.21.15.86:0/178499719 learned_addr learned my addr 172.21.15.86:0/178499719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:05.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5400700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e8103fc0 0x7fb6e806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5e00700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102e70 0x7fb6e806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5e00700 1 -- 172.21.15.86:0/178499719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8104980 msgr2=0x7fb6e81a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5e00700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8104980 0x7fb6e81a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5e00700 1 -- 172.21.15.86:0/178499719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e8103fc0 msgr2=0x7fb6e806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5e00700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e8103fc0 0x7fb6e806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.183+0000 7fb6e5e00700 1 -- 172.21.15.86:0/178499719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0009d00 con 0x7fb6e8102e70 2024-03-22T19:13:05.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.188+0000 7fb6e5e00700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102e70 0x7fb6e806f7a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d950 tx=0x7fb6d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:05.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.189+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8009930 con 0x7fb6e8102e70 2024-03-22T19:13:05.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.189+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e81a9870 con 0x7fb6e8102e70 2024-03-22T19:13:05.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.189+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e81a9dd0 con 0x7fb6e8102e70 2024-03-22T19:13:05.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.189+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d8010460 con 0x7fb6e8102e70 2024-03-22T19:13:05.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.189+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800b5d0 con 0x7fb6e8102e70 2024-03-22T19:13:05.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.191+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c8005320 con 0x7fb6e8102e70 2024-03-22T19:13:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.193+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fb6d8009a90 con 0x7fb6e8102e70 2024-03-22T19:13:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.195+0000 7fb6dea00700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6cc0a0a90 0x7fb6cc0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.195+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb6d80cbb60 con 0x7fb6e8102e70 2024-03-22T19:13:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.195+0000 7fb6e5400700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6cc0a0a90 0x7fb6cc0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.196+0000 7fb6e5400700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6cc0a0a90 0x7fb6cc0a2f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009a10 tx=0x7fb6d4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:05.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.202+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb6d8095520 con 0x7fb6e8102e70 2024-03-22T19:13:05.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:05 smithi167 ceph-mon[47728]: pgmap v349: 97 pgs: 97 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 239 MiB/s wr, 89 op/s 2024-03-22T19:13:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.619+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6c8005190 con 0x7fb6e8102e70 2024-03-22T19:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.620+0000 7fb6dea00700 1 -- 172.21.15.86:0/178499719 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10927 (secure 0 0 0) 0x7fb6d8021020 con 0x7fb6e8102e70 2024-03-22T19:13:05.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:13:05.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:12:48.181207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.623+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6cc0a0a90 msgr2=0x7fb6cc0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.623+0000 7fb6ed200700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6cc0a0a90 0x7fb6cc0a2f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009a10 tx=0x7fb6d4009190 comp rx=0 tx=0).stop 2024-03-22T19:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.623+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102e70 msgr2=0x7fb6e806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.623+0000 7fb6ed200700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102e70 0x7fb6e806f7a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d950 tx=0x7fb6d800dc60 comp rx=0 tx=0).stop 2024-03-22T19:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 shutdown_connections 2024-03-22T19:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb6cc0a0a90 0x7fb6cc0a2f50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6e8104980 0x7fb6e81a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102e70 0x7fb6e806f7a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 --2- 172.21.15.86:0/178499719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e8103fc0 0x7fb6e806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 >> 172.21.15.86:0/178499719 conn(0x7fb6e80fe420 msgr2=0x7fb6e8107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 shutdown_connections 2024-03-22T19:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.624+0000 7fb6ed200700 1 -- 172.21.15.86:0/178499719 wait complete. 2024-03-22T19:13:05.629 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-03-22T19:13:05.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:05 smithi086 ceph-mon[38032]: pgmap v349: 97 pgs: 97 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 239 MiB/s wr, 89 op/s 2024-03-22T19:13:05.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:13:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:05 smithi131 ceph-mon[78559]: pgmap v349: 97 pgs: 97 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 239 MiB/s wr, 89 op/s 2024-03-22T19:13:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.969+0000 7f7cbce00700 1 -- 172.21.15.86:0/3377653581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 msgr2=0x7f7cb8109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.969+0000 7f7cbce00700 1 --2- 172.21.15.86:0/3377653581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 0x7f7cb8109180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009f90 tx=0x7f7cb000b330 comp rx=0 tx=0).stop 2024-03-22T19:13:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 -- 172.21.15.86:0/3377653581 shutdown_connections 2024-03-22T19:13:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 --2- 172.21.15.86:0/3377653581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 0x7f7cb8109180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 --2- 172.21.15.86:0/3377653581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 --2- 172.21.15.86:0/3377653581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb8101f30 0x7f7cb8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 -- 172.21.15.86:0/3377653581 >> 172.21.15.86:0/3377653581 conn(0x7f7cb80fc2f0 msgr2=0x7f7cb80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 -- 172.21.15.86:0/3377653581 shutdown_connections 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.970+0000 7f7cbce00700 1 -- 172.21.15.86:0/3377653581 wait complete. 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.971+0000 7f7cbce00700 1 Processor -- start 2024-03-22T19:13:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.971+0000 7f7cbce00700 1 -- start start 2024-03-22T19:13:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cbce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cbce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb8101f30 0x7f7cb8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cbce00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 0x7f7cb81a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cbce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a0b10 con 0x7f7cb8101f30 2024-03-22T19:13:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cbce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a0c50 con 0x7f7cb8100d40 2024-03-22T19:13:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cbce00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a0de0 con 0x7f7cb81028d0 2024-03-22T19:13:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cb5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cb5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:47900/0 (socket says 172.21.15.86:47900) 2024-03-22T19:13:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.972+0000 7f7cb5e00700 1 -- 172.21.15.86:0/2769975134 learned_addr learned my addr 172.21.15.86:0/2769975134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7cb5e00700 1 -- 172.21.15.86:0/2769975134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 msgr2=0x7f7cb81a0540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:13:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 0x7f7cb81a0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7cb5e00700 1 -- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb8101f30 msgr2=0x7f7cb8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb8101f30 0x7f7cb8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7cb5e00700 1 -- 172.21.15.86:0/2769975134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0009c40 con 0x7f7cb8100d40 2024-03-22T19:13:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8198fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e9f0 tx=0x7f7ca800edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7caea00700 1 -- 172.21.15.86:0/2769975134 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800cbd0 con 0x7f7cb8100d40 2024-03-22T19:13:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7caea00700 1 -- 172.21.15.86:0/2769975134 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca80044e0 con 0x7f7cb8100d40 2024-03-22T19:13:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.973+0000 7f7caea00700 1 -- 172.21.15.86:0/2769975134 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8003d90 con 0x7f7cb8100d40 2024-03-22T19:13:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.974+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb81032f0 con 0x7f7cb8100d40 2024-03-22T19:13:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.974+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb81037b0 con 0x7f7cb8100d40 2024-03-22T19:13:05.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.978+0000 7f7caea00700 1 -- 172.21.15.86:0/2769975134 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f7ca8003f30 con 0x7f7cb8100d40 2024-03-22T19:13:05.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 Processor -- start 2024-03-22T19:13:05.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 -- start start 2024-03-22T19:13:05.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 0x55fa1b9f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b94c990 con 0x55fa1b9f5420 2024-03-22T19:13:05.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b94a350 con 0x55fa1b9f4580 2024-03-22T19:13:05.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.980+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b945c40 con 0x55fa1b9f3c20 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7caea00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c9c0a0b10 0x7f7c9c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 0x55fa1b9f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 0x55fa1b9f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:47910/0 (socket says 172.21.15.86:47910) 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47102/0 (socket says 172.21.15.86:47102) 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1607996020 learned_addr learned my addr 172.21.15.86:0/1607996020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7caea00700 1 -- 172.21.15.86:0/2769975134 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7ca8014070 con 0x7f7cb8100d40 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb6800700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c9c0a0b10 0x7f7c9c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1607996020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 msgr2=0x55fa1b9f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1607996020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 msgr2=0x55fa1b9f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.981+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1607996020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa1b7c73b0 con 0x55fa1b9f4580 2024-03-22T19:13:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.982+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 0x55fa1b9f4e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80a1990 tx=0x7f7ca806bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2734b9795052487 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.982+0000 7f7ca3400700 1 -- 172.21.15.86:0/1607996020 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca809a140 con 0x55fa1b9f4580 2024-03-22T19:13:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.982+0000 7f7ca3400700 1 -- 172.21.15.86:0/1607996020 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca8016330 con 0x55fa1b9f4580 2024-03-22T19:13:05.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.982+0000 7f7ca3400700 1 -- 172.21.15.86:0/1607996020 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80a1ec0 con 0x55fa1b9f4580 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1607996020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 msgr2=0x55fa1b9f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cb6800700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c9c0a0b10 0x7f7c9c0a2fd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005bd0 tx=0x7f7cb0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 0x55fa1b9f4e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80a1990 tx=0x7f7ca806bdb0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1607996020 shutdown_connections 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1607996020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f4580 0x55fa1b9f4e40 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80a1990 tx=0x7f7ca806bdb0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1607996020 >> 172.21.15.86:0/1607996020 conn(0x55fa1b969220 msgr2=0x55fa1b9eebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1607996020 shutdown_connections 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1607996020 wait complete. 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.983+0000 7f7cd0955b80 1 -- start start 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 Processor -- start 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b8e1d80 con 0x55fa1b9f5420 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b8e1a20 con 0x55fa1b9f3c20 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b7c96a0 con 0x55fa1ba03200 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cd0955b80 10 client.0 _ll_register_callbacks cb 0x55fa1b74d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cb6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cb5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cb6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47104/0 (socket says 172.21.15.86:47104) 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.984+0000 7f7cb6800700 1 -- 172.21.15.86:0/578146521 learned_addr learned my addr 172.21.15.86:0/578146521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb6800700 1 -- 172.21.15.86:0/578146521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f3c20 msgr2=0x55fa1b9fdd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb6800700 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9fdd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb6800700 1 -- 172.21.15.86:0/578146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 msgr2=0x55fa1b9fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb6800700 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb6800700 1 -- 172.21.15.86:0/578146521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb81037b0 con 0x55fa1ba03200 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb5400700 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7cb6800700 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00077f0 tx=0x7f7cb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6531124f5cc5007 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.985+0000 7f7c97e00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7c97e00700 1 -- 172.21.15.86:0/578146521 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb000e040 con 0x55fa1ba03200 2024-03-22T19:13:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa1b9e6ad0 con 0x55fa1ba03200 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7c97e00700 1 -- 172.21.15.86:0/578146521 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7cb002a870 con 0x55fa1ba03200 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7c97e00700 1 -- 172.21.15.86:0/578146521 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb002aa30 con 0x55fa1ba03200 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7cd0955b80 10 client.25855 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fa1b773b60 con 0x55fa1ba03200 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7c97e00700 1 -- 172.21.15.86:0/578146521 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cb0038020 con 0x55fa1ba03200 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7c97e00700 1 -- 172.21.15.86:0/578146521 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cb001fbd0 con 0x55fa1ba03200 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7cd0955b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.986+0000 7f7cd0955b80 10 client.25855 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7cd0955b80 20 client.25855 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba09d90 0x55fa1ba0c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7cd0955b80 4 client.25855 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa1b93e700 con 0x55fa1ba09d90 2024-03-22T19:13:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba09d90 0x55fa1ba0c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba09d90 0x55fa1ba0c1b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d869b4fea654240 server_cookie=bc96db4cb5d699e5 in_seq=0 out_seq=0 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.987+0000 7f7c97e00700 10 client.25855 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7c97e00700 1 -- 172.21.15.86:0/578146521 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fa1b93e700 con 0x55fa1ba09d90 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7c97e00700 10 client.25855 handle_command_reply: tid=0 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 client.25855 shutdown 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 msgr2=0x55fa1ba055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00077f0 tx=0x7f7cb000b330 comp rx=0 tx=0).stop 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 shutdown_connections 2024-03-22T19:13:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba09d90 0x55fa1ba0c1b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f3c20 0x55fa1b9fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7c97e00700 0 client.25855 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/578146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f5420 0x55fa1b9fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 >> 172.21.15.86:0/578146521 conn(0x55fa1b969220 msgr2=0x55fa1b9ee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 shutdown_connections 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.988+0000 7f7cd0955b80 1 -- 172.21.15.86:0/578146521 wait complete. 2024-03-22T19:13:05.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:05.989+0000 7f7cd0955b80 20 client.25855 trim_cache size 0 max 16384 2024-03-22T19:13:06.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 Processor -- start 2024-03-22T19:13:06.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 -- start start 2024-03-22T19:13:06.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1ba10ef0 con 0x55fa1ba03200 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b9f4a20 con 0x55fa1b9f5420 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1b9fc4d0 con 0x55fa1b9f3c20 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cb5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cb5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba196b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:47924/0 (socket says 172.21.15.86:47924) 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cb5400700 1 -- 172.21.15.86:0/1546380045 learned_addr learned my addr 172.21.15.86:0/1546380045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.002+0000 7f7cb6800700 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5400700 1 -- 172.21.15.86:0/1546380045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 msgr2=0x55fa1ba05cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5400700 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba05cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5400700 1 -- 172.21.15.86:0/1546380045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1ba03200 msgr2=0x55fa1ba055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5400700 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5400700 1 -- 172.21.15.86:0/1546380045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa1b90bea0 con 0x55fa1b9f5420 2024-03-22T19:13:06.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:06.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7cb5400700 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba196b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40037b0 tx=0x7f7ca4003ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53de2918ed4108da server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7ca3e00700 1 -- 172.21.15.86:0/1546380045 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4016070 con 0x55fa1b9f5420 2024-03-22T19:13:06.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7ca3e00700 1 -- 172.21.15.86:0/1546380045 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca4004cf0 con 0x55fa1b9f5420 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.003+0000 7f7ca3e00700 1 -- 172.21.15.86:0/1546380045 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400b940 con 0x55fa1b9f5420 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1546380045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 msgr2=0x55fa1ba196b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba196b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40037b0 tx=0x7f7ca4003ac0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1546380045 shutdown_connections 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1ba03200 0x55fa1ba055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1546380045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba196b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1546380045 >> 172.21.15.86:0/1546380045 conn(0x55fa1b969220 msgr2=0x55fa1b9ee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1546380045 shutdown_connections 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.004+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1546380045 wait complete. 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.005+0000 7f7cd0955b80 1 -- start start 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.005+0000 7f7cd0955b80 1 Processor -- start 2024-03-22T19:13:06.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba18480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1ba189c0 con 0x55fa1b9f3c20 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1ba18b30 con 0x55fa1b9f5420 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1ba18cf0 con 0x55fa1ba03200 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cd0955b80 10 client.0 _ll_register_callbacks cb 0x55fa1b74d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cb5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cb5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba170a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38500/0 (socket says 172.21.15.86:38500) 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1346438615 learned_addr learned my addr 172.21.15.86:0/1346438615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cb6800700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba18480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.006+0000 7f7cb5400700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 msgr2=0x55fa1ba18480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba18480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1346438615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 msgr2=0x55fa1ba175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb5e00700 1 -- 172.21.15.86:0/1346438615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa1b7c73b0 con 0x55fa1b9f3c20 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb6800700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba18480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7cb5e00700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba170a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8095f40 tx=0x7f7ca8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c23f3c9ca5318 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.007+0000 7f7c97e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.008+0000 7f7c97e00700 1 -- 172.21.15.86:0/1346438615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80a1ec0 con 0x55fa1b9f3c20 2024-03-22T19:13:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.008+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ca8016330 con 0x55fa1b9f3c20 2024-03-22T19:13:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.008+0000 7f7c97e00700 1 -- 172.21.15.86:0/1346438615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7ca8099a40 con 0x55fa1b9f3c20 2024-03-22T19:13:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.008+0000 7f7c97e00700 1 -- 172.21.15.86:0/1346438615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca809d870 con 0x55fa1b9f3c20 2024-03-22T19:13:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.009+0000 7f7cd0955b80 10 client.15831 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.009+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f7ca40044e0 con 0x55fa1b9f3c20 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.009+0000 7f7c97e00700 1 -- 172.21.15.86:0/1346438615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca8016330 con 0x55fa1b9f3c20 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.009+0000 7f7c97e00700 1 -- 172.21.15.86:0/1346438615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7ca8098f30 con 0x55fa1b9f3c20 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.009+0000 7f7cd0955b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.010+0000 7f7cd0955b80 10 client.15831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.010+0000 7f7cd0955b80 20 client.15831 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.010+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba5c490 0x55fa1ba5e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.010+0000 7f7cd0955b80 4 client.15831 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.010+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa1b794cf0 con 0x55fa1ba5c490 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.010+0000 7f7cb5400700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba5c490 0x55fa1ba5e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cb5400700 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba5c490 0x55fa1ba5e8b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c9af3920f70cc08 server_cookie=3e5caac3722c2a47 in_seq=0 out_seq=0 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7c97e00700 10 client.15831 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7c97e00700 1 -- 172.21.15.86:0/1346438615 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa1b794cf0 con 0x55fa1ba5c490 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7c97e00700 10 client.15831 handle_command_reply: tid=0 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 client.15831 shutdown 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 msgr2=0x55fa1ba170a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba170a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8095f40 tx=0x7f7ca8009290 comp rx=0 tx=0).stop 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 shutdown_connections 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55fa1ba5c490 0x55fa1ba5e8b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa1ba03200 0x55fa1ba18480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa1b9f3c20 0x55fa1ba170a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 --2- 172.21.15.86:0/1346438615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa1b9f5420 0x55fa1ba175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7c97e00700 0 client.15831 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.011+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 >> 172.21.15.86:0/1346438615 conn(0x55fa1b969220 msgr2=0x55fa1b9eaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.012+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 shutdown_connections 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.012+0000 7f7cd0955b80 1 -- 172.21.15.86:0/1346438615 wait complete. 2024-03-22T19:13:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.012+0000 7f7cd0955b80 20 client.15831 trim_cache size 0 max 16384 2024-03-22T19:13:06.016 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.012+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c9c0a0b10 msgr2=0x7f7c9c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.012+0000 7f7cbce00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c9c0a0b10 0x7f7c9c0a2fd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005bd0 tx=0x7f7cb0009920 comp rx=0 tx=0).stop 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 msgr2=0x7f7cb8198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8198fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e9f0 tx=0x7f7ca800edb0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 shutdown_connections 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7c9c0a0b10 0x7f7c9c0a2fd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb81028d0 0x7f7cb81a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb8100d40 0x7f7cb8198fe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 --2- 172.21.15.86:0/2769975134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb8101f30 0x7f7cb8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 >> 172.21.15.86:0/2769975134 conn(0x7f7cb80fc2f0 msgr2=0x7f7cb8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 shutdown_connections 2024-03-22T19:13:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.013+0000 7f7cbce00700 1 -- 172.21.15.86:0/2769975134 wait complete. 2024-03-22T19:13:06.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:13:06.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 -- 172.21.15.86:0/3153344555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 msgr2=0x7f8154073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 --2- 172.21.15.86:0/3153344555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f8154073c90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8140009990 tx=0x7f8140009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 -- 172.21.15.86:0/3153344555 shutdown_connections 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 --2- 172.21.15.86:0/3153344555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f815410d310 0x7f815410f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 --2- 172.21.15.86:0/3153344555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 0x7f8154073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 --2- 172.21.15.86:0/3153344555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f8154073c90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 -- 172.21.15.86:0/3153344555 >> 172.21.15.86:0/3153344555 conn(0x7f81540fc2b0 msgr2=0x7f81540fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 -- 172.21.15.86:0/3153344555 shutdown_connections 2024-03-22T19:13:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.379+0000 7f8159a00700 1 -- 172.21.15.86:0/3153344555 wait complete. 2024-03-22T19:13:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.381+0000 7f8159a00700 1 Processor -- start 2024-03-22T19:13:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.381+0000 7f8159a00700 1 -- start start 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8159a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f815419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8159a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 0x7f815419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8159a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f815410d310 0x7f81541a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8152a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f815419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8152a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f815419d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:47950/0 (socket says 172.21.15.86:47950) 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8152a00700 1 -- 172.21.15.86:0/96609184 learned_addr learned my addr 172.21.15.86:0/96609184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.382+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541a4e30 con 0x7f8154074df0 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541a4f70 con 0x7f8154073810 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541a5100 con 0x7f815410d310 2024-03-22T19:13:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8153400700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f815410d310 0x7f81541a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8152000700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 0x7f815419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8152a00700 1 -- 172.21.15.86:0/96609184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f815410d310 msgr2=0x7f81541a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8152a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f815410d310 0x7f81541a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8152a00700 1 -- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 msgr2=0x7f815419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8152a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 0x7f815419d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.383+0000 7f8152a00700 1 -- 172.21.15.86:0/96609184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400095f0 con 0x7f8154073810 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.384+0000 7f8152000700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 0x7f815419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.384+0000 7f8152a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f815419d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f814400d980 tx=0x7f814400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.384+0000 7f814b400700 1 -- 172.21.15.86:0/96609184 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144004220 con 0x7f8154073810 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.384+0000 7f814b400700 1 -- 172.21.15.86:0/96609184 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8144004cf0 con 0x7f8154073810 2024-03-22T19:13:06.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.384+0000 7f814b400700 1 -- 172.21.15.86:0/96609184 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400b750 con 0x7f8154073810 2024-03-22T19:13:06.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.385+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81541a53e0 con 0x7f8154073810 2024-03-22T19:13:06.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.386+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541a5870 con 0x7f8154073810 2024-03-22T19:13:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.392+0000 7f814b400700 1 -- 172.21.15.86:0/96609184 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8144004750 con 0x7f8154073810 2024-03-22T19:13:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.392+0000 7f816d48cb80 1 Processor -- start 2024-03-22T19:13:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.394+0000 7f814b400700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81380a0b10 0x7f81380a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.394+0000 7f814b400700 1 -- 172.21.15.86:0/96609184 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f81440cbbc0 con 0x7f8154073810 2024-03-22T19:13:06.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 -- start start 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 0x5613ed78d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78e420 0x5613ed78e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f8153400700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81380a0b10 0x7f81380a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed6e5990 con 0x5613ed78d580 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed6e3350 con 0x5613ed78e420 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.395+0000 7f816d48cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed6dec40 con 0x5613ed78cc20 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 0x5613ed78d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 0x5613ed78d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47152/0 (socket says 172.21.15.86:47152) 2024-03-22T19:13:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 -- 172.21.15.86:0/2982075126 learned_addr learned my addr 172.21.15.86:0/2982075126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8153400700 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78e420 0x5613ed78e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152000700 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 -- 172.21.15.86:0/2982075126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78e420 msgr2=0x5613ed78e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78e420 0x5613ed78e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 -- 172.21.15.86:0/2982075126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 msgr2=0x5613ed78de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152a00700 1 -- 172.21.15.86:0/2982075126 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613ed5603b0 con 0x5613ed78cc20 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.396+0000 7f8152000700 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.397+0000 7f8152a00700 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 0x5613ed78d040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81440a1230 tx=0x7f8144098ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf2ce7687da17f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.397+0000 7f8153400700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81380a0b10 0x7f81380a2fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f814c00f580 tx=0x7f814c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.397+0000 7f813fe00700 1 -- 172.21.15.86:0/2982075126 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440999e0 con 0x5613ed78cc20 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.397+0000 7f813fe00700 1 -- 172.21.15.86:0/2982075126 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81440992e0 con 0x5613ed78cc20 2024-03-22T19:13:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.397+0000 7f813fe00700 1 -- 172.21.15.86:0/2982075126 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144097780 con 0x5613ed78cc20 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 -- 172.21.15.86:0/2982075126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 msgr2=0x5613ed78d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 0x5613ed78d040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81440a1230 tx=0x7f8144098ea0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 -- 172.21.15.86:0/2982075126 shutdown_connections 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78e420 0x5613ed78e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78cc20 0x5613ed78d040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 --2- 172.21.15.86:0/2982075126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.398+0000 7f816d48cb80 1 -- 172.21.15.86:0/2982075126 >> 172.21.15.86:0/2982075126 conn(0x5613ed702220 msgr2=0x5613ed787bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.399+0000 7f816d48cb80 1 -- 172.21.15.86:0/2982075126 shutdown_connections 2024-03-22T19:13:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.399+0000 7f816d48cb80 1 -- 172.21.15.86:0/2982075126 wait complete. 2024-03-22T19:13:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.400+0000 7f816d48cb80 1 -- start start 2024-03-22T19:13:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.400+0000 7f816d48cb80 1 Processor -- start 2024-03-22T19:13:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78cc20 0x5613ed7a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78d580 0x5613ed7a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed7aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed67ad80 con 0x5613ed78cc20 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed67aa20 con 0x5613ed78d580 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed5626a0 con 0x5613ed78e420 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed7aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed7aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47154/0 (socket says 172.21.15.86:47154) 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 -- 172.21.15.86:0/940518420 learned_addr learned my addr 172.21.15.86:0/940518420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f816d48cb80 10 client.0 _ll_register_callbacks cb 0x5613ed4e63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 -- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78d580 msgr2=0x5613ed7a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8152000700 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78d580 0x5613ed7a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78d580 0x5613ed7a7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 -- 172.21.15.86:0/940518420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78cc20 msgr2=0x5613ed7a7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78cc20 0x5613ed7a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.401+0000 7f8153400700 1 -- 172.21.15.86:0/940518420 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541a5870 con 0x5613ed78e420 2024-03-22T19:13:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f8153400700 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed7aefd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f814c017e10 tx=0x7f814c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b347e14d9b1e0b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f813cc00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f813cc00700 1 -- 172.21.15.86:0/940518420 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00e040 con 0x5613ed78e420 2024-03-22T19:13:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f813cc00700 1 -- 172.21.15.86:0/940518420 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f814c00fe40 con 0x5613ed78e420 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613ed77fad0 con 0x5613ed78e420 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f813cc00700 1 -- 172.21.15.86:0/940518420 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f814c015e50 con 0x5613ed78e420 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f816d48cb80 10 client.25873 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.402+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5613ed50cb60 con 0x5613ed78e420 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f813cc00700 1 -- 172.21.15.86:0/940518420 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613ed77fad0 con 0x5613ed78e420 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f813cc00700 1 -- 172.21.15.86:0/940518420 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f814c01f040 con 0x5613ed78e420 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f816d48cb80 10 client.25873 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f816d48cb80 10 client.25873 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f816d48cb80 20 client.25873 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f816d48cb80 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed7b37f0 0x5613ed7b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f816d48cb80 4 client.25873 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613ed6d7700 con 0x5613ed7b37f0 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.403+0000 7f8152a00700 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed7b37f0 0x5613ed7b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.404+0000 7f8152a00700 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed7b37f0 0x5613ed7b5c10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db68e8b99b6de551 server_cookie=573da5cec3ac3aec in_seq=0 out_seq=0 2024-03-22T19:13:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.404+0000 7f813cc00700 10 client.25873 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f813cc00700 1 -- 172.21.15.86:0/940518420 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x7f81440b1e90 con 0x5613ed7b37f0 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f813cc00700 10 client.25873 handle_command_reply: tid=0 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 client.25873 shutdown 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 msgr2=0x5613ed7aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed7aefd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f814c017e10 tx=0x7f814c004ad0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 shutdown_connections 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed7b37f0 0x5613ed7b5c10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed7aefd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f813cc00700 0 client.25873 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78cc20 0x5613ed7a7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 --2- 172.21.15.86:0/940518420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78d580 0x5613ed7a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 >> 172.21.15.86:0/940518420 conn(0x5613ed702220 msgr2=0x5613ed7871f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.405+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 shutdown_connections 2024-03-22T19:13:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.406+0000 7f816d48cb80 1 -- 172.21.15.86:0/940518420 wait complete. 2024-03-22T19:13:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.406+0000 7f816d48cb80 20 client.25873 trim_cache size 0 max 16384 2024-03-22T19:13:06.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.424+0000 7f816d48cb80 1 Processor -- start 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.424+0000 7f816d48cb80 1 -- start start 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed794990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f816d48cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed7a64e0 con 0x5613ed78d580 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f816d48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed77f610 con 0x5613ed78cc20 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f816d48cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed794610 con 0x5613ed78e420 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f8152a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f8152a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:47990/0 (socket says 172.21.15.86:47990) 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f8152a00700 1 -- 172.21.15.86:0/1773587528 learned_addr learned my addr 172.21.15.86:0/1773587528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f8153400700 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed794990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.425+0000 7f8152000700 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8152a00700 1 -- 172.21.15.86:0/1773587528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 msgr2=0x5613ed794990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8152a00700 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed794990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8152a00700 1 -- 172.21.15.86:0/1773587528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 msgr2=0x5613ed78e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8152a00700 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8152a00700 1 -- 172.21.15.86:0/1773587528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613ed6a4ea0 con 0x5613ed78cc20 2024-03-22T19:13:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8152a00700 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7acf80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8144095d50 tx=0x7f8144000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ad2e0c20a2d1cd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8148c00700 1 -- 172.21.15.86:0/1773587528 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440b3ea0 con 0x5613ed78cc20 2024-03-22T19:13:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8148c00700 1 -- 172.21.15.86:0/1773587528 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81440b1e90 con 0x5613ed78cc20 2024-03-22T19:13:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.426+0000 7f8148c00700 1 -- 172.21.15.86:0/1773587528 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400e350 con 0x5613ed78cc20 2024-03-22T19:13:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 -- 172.21.15.86:0/1773587528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 msgr2=0x5613ed7acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7acf80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8144095d50 tx=0x7f8144000bc0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 -- 172.21.15.86:0/1773587528 shutdown_connections 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78e420 0x5613ed794990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7acf80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 --2- 172.21.15.86:0/1773587528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78d580 0x5613ed78e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.427+0000 7f816d48cb80 1 -- 172.21.15.86:0/1773587528 >> 172.21.15.86:0/1773587528 conn(0x5613ed702220 msgr2=0x5613ed7a6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.428+0000 7f816d48cb80 1 -- 172.21.15.86:0/1773587528 shutdown_connections 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.428+0000 7f816d48cb80 1 -- 172.21.15.86:0/1773587528 wait complete. 2024-03-22T19:13:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.428+0000 7f816d48cb80 1 -- start start 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.428+0000 7f816d48cb80 1 Processor -- start 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78d580 0x5613ed7fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 0x5613ed8048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed804df0 con 0x5613ed78e420 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed804f30 con 0x5613ed78cc20 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ed8050c0 con 0x5613ed78d580 2024-03-22T19:13:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.429+0000 7f816d48cb80 10 client.0 _ll_register_callbacks cb 0x7f814c0177e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8153400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 0x5613ed8048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8153400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 0x5613ed8048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38530/0 (socket says 172.21.15.86:38530) 2024-03-22T19:13:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8153400700 1 -- 172.21.15.86:0/480765701 learned_addr learned my addr 172.21.15.86:0/480765701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8152000700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78d580 0x5613ed7fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8152000700 1 -- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 msgr2=0x5613ed7fd4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8152000700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7fd4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8152000700 1 -- 172.21.15.86:0/480765701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 msgr2=0x5613ed8048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8152000700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 0x5613ed8048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8152000700 1 -- 172.21.15.86:0/480765701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613ed5603b0 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.430+0000 7f8153400700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 0x5613ed8048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f8152000700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78d580 0x5613ed7fda30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f81400098e0 tx=0x7f81400041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41558a9fa8aafa38 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f813cc00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f813cc00700 1 -- 172.21.15.86:0/480765701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140003680 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f813cc00700 1 -- 172.21.15.86:0/480765701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f81400037e0 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81440992e0 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f813cc00700 1 -- 172.21.15.86:0/480765701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81400039a0 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.431+0000 7f816d48cb80 10 client.25879 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5613ed564cd0 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f813cc00700 1 -- 172.21.15.86:0/480765701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81440992e0 con 0x5613ed78d580 2024-03-22T19:13:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f813cc00700 1 -- 172.21.15.86:0/480765701 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8140024d20 con 0x5613ed78d580 2024-03-22T19:13:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 10 client.25879 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 10 client.25879 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:13:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 20 client.25879 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed806a40 0x5613ed808e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 4 client.25879 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.432+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5613ed52dcf0 con 0x5613ed806a40 2024-03-22T19:13:06.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.433+0000 7f8152a00700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed806a40 0x5613ed808e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.433+0000 7f8152a00700 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed806a40 0x5613ed808e60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b885188b7221117 server_cookie=48fc6675c1650fed in_seq=0 out_seq=0 2024-03-22T19:13:06.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.433+0000 7f813cc00700 10 client.25879 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:06.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.433+0000 7f813cc00700 1 -- 172.21.15.86:0/480765701 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5613ed52dcf0 con 0x5613ed806a40 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f813cc00700 10 client.25879 handle_command_reply: tid=0 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 client.25879 shutdown 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78d580 msgr2=0x5613ed7fda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78d580 0x5613ed7fda30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f81400098e0 tx=0x7f81400041f0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 shutdown_connections 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613ed78e420 0x5613ed8048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613ed78cc20 0x5613ed7fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5613ed806a40 0x5613ed808e60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 --2- 172.21.15.86:0/480765701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5613ed78d580 0x5613ed7fda30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 >> 172.21.15.86:0/480765701 conn(0x5613ed702220 msgr2=0x5613ed787aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f813cc00700 0 client.25879 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 shutdown_connections 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 1 -- 172.21.15.86:0/480765701 wait complete. 2024-03-22T19:13:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.434+0000 7f816d48cb80 20 client.25879 trim_cache size 0 max 16384 2024-03-22T19:13:06.438 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81380a0b10 msgr2=0x7f81380a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81380a0b10 0x7f81380a2fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f814c00f580 tx=0x7f814c00b040 comp rx=0 tx=0).stop 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 msgr2=0x7f815419d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f815419d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f814400d980 tx=0x7f814400dc90 comp rx=0 tx=0).stop 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 shutdown_connections 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f81380a0b10 0x7f81380a2fd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f815410d310 0x7f81541a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8154073810 0x7f815419d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 --2- 172.21.15.86:0/96609184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154074df0 0x7f815419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.435+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 >> 172.21.15.86:0/96609184 conn(0x7f81540fc2b0 msgr2=0x7f81541006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.436+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 shutdown_connections 2024-03-22T19:13:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.436+0000 7f8159a00700 1 -- 172.21.15.86:0/96609184 wait complete. 2024-03-22T19:13:06.466 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:06 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/178499719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:13:06.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:13:06.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:06 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/178499719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:13:06.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:06 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/178499719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 -- 172.21.15.86:0/952886465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108520 msgr2=0x7f64441089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 --2- 172.21.15.86:0/952886465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108520 0x7f64441089a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6430009990 tx=0x7f6430009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 -- 172.21.15.86:0/952886465 shutdown_connections 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 --2- 172.21.15.86:0/952886465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6444108ee0 0x7f644410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 --2- 172.21.15.86:0/952886465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f6444072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 --2- 172.21.15.86:0/952886465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108520 0x7f64441089a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.788+0000 7f6449c00700 1 -- 172.21.15.86:0/952886465 >> 172.21.15.86:0/952886465 conn(0x7f644406dd20 msgr2=0x7f6444070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.789+0000 7f6449c00700 1 -- 172.21.15.86:0/952886465 shutdown_connections 2024-03-22T19:13:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.790+0000 7f6449c00700 1 -- 172.21.15.86:0/952886465 wait complete. 2024-03-22T19:13:06.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.804+0000 7f6449c00700 1 Processor -- start 2024-03-22T19:13:06.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.806+0000 7f6449c00700 1 -- start start 2024-03-22T19:13:06.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.808+0000 7f6449c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f64441a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.808+0000 7f6449c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6444108520 0x7f64441a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6449c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108ee0 0x7f64441acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6449c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64441ad550 con 0x7f64440721e0 2024-03-22T19:13:06.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6449c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64441ad690 con 0x7f6444108520 2024-03-22T19:13:06.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6449c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64441ad820 con 0x7f6444108ee0 2024-03-22T19:13:06.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6442a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f64441a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6442a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f64441a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38536/0 (socket says 172.21.15.86:38536) 2024-03-22T19:13:06.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.809+0000 7f6442a00700 1 -- 172.21.15.86:0/4059604399 learned_addr learned my addr 172.21.15.86:0/4059604399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.810+0000 7f6442000700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6444108520 0x7f64441a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.810+0000 7f6443400700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108ee0 0x7f64441acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.810+0000 7f6442a00700 1 -- 172.21.15.86:0/4059604399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108ee0 msgr2=0x7f64441acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.810+0000 7f6442a00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108ee0 0x7f64441acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.810+0000 7f6442a00700 1 -- 172.21.15.86:0/4059604399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6444108520 msgr2=0x7f64441a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.810+0000 7f6442a00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6444108520 0x7f64441a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.811+0000 7f6442a00700 1 -- 172.21.15.86:0/4059604399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300095f0 con 0x7f64440721e0 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.811+0000 7f6443400700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108ee0 0x7f64441acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:06.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.812+0000 7f6442a00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f64441a5a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f643400cce0 tx=0x7f64340074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.812+0000 7f643b400700 1 -- 172.21.15.86:0/4059604399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434007cf0 con 0x7f64440721e0 2024-03-22T19:13:06.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.812+0000 7f643b400700 1 -- 172.21.15.86:0/4059604399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643400ce80 con 0x7f64440721e0 2024-03-22T19:13:06.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.812+0000 7f643b400700 1 -- 172.21.15.86:0/4059604399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434018660 con 0x7f64440721e0 2024-03-22T19:13:06.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.812+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64441adb00 con 0x7f64440721e0 2024-03-22T19:13:06.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.813+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64441adfc0 con 0x7f64440721e0 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.819+0000 7f643b400700 1 -- 172.21.15.86:0/4059604399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6434004870 con 0x7f64440721e0 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.823+0000 7f645d702b80 1 Processor -- start 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.823+0000 7f643b400700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64280a0c90 0x7f64280a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.823+0000 7f645d702b80 1 -- start start 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.823+0000 7f643b400700 1 -- 172.21.15.86:0/4059604399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f64340cc080 con 0x7f64440721e0 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.823+0000 7f645d702b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.823+0000 7f645d702b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe725e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.824+0000 7f645d702b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe7268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.824+0000 7f645d702b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe67d990 con 0x55cdbe726420 2024-03-22T19:13:06.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.824+0000 7f645d702b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe67b350 con 0x55cdbe724c20 2024-03-22T19:13:06.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.824+0000 7f645d702b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe676c40 con 0x55cdbe725580 2024-03-22T19:13:06.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.825+0000 7f6443400700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64280a0c90 0x7f64280a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.826+0000 7f6442a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.826+0000 7f6442000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe725e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.827+0000 7f6442a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48014/0 (socket says 172.21.15.86:48014) 2024-03-22T19:13:06.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.827+0000 7f6442000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe725e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47198/0 (socket says 172.21.15.86:47198) 2024-03-22T19:13:06.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.827+0000 7f6442a00700 1 -- 172.21.15.86:0/53218089 learned_addr learned my addr 172.21.15.86:0/53218089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.827+0000 7f6443400700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64280a0c90 0x7f64280a3150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f643c009ca0 tx=0x7f643c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6443400700 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe7268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6442000700 1 -- 172.21.15.86:0/53218089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 msgr2=0x55cdbe725040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6442000700 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6442000700 1 -- 172.21.15.86:0/53218089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 msgr2=0x55cdbe7268a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6442000700 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe7268a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6442000700 1 -- 172.21.15.86:0/53218089 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbe4f83b0 con 0x55cdbe725580 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.828+0000 7f6442000700 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe725e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6430003790 tx=0x7f6430009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb66faa5de2f20d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.829+0000 7f642fe00700 1 -- 172.21.15.86:0/53218089 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000b9d0 con 0x55cdbe725580 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.829+0000 7f642fe00700 1 -- 172.21.15.86:0/53218089 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643001e040 con 0x55cdbe725580 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.829+0000 7f642fe00700 1 -- 172.21.15.86:0/53218089 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430005620 con 0x55cdbe725580 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.829+0000 7f645d702b80 1 -- 172.21.15.86:0/53218089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 msgr2=0x55cdbe725e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.829+0000 7f645d702b80 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe725e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6430003790 tx=0x7f6430009ee0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 -- 172.21.15.86:0/53218089 shutdown_connections 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe7268a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 --2- 172.21.15.86:0/53218089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe725e40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 -- 172.21.15.86:0/53218089 >> 172.21.15.86:0/53218089 conn(0x55cdbe69a220 msgr2=0x55cdbe71fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 -- 172.21.15.86:0/53218089 shutdown_connections 2024-03-22T19:13:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.830+0000 7f645d702b80 1 -- 172.21.15.86:0/53218089 wait complete. 2024-03-22T19:13:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.831+0000 7f645d702b80 1 -- start start 2024-03-22T19:13:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.831+0000 7f645d702b80 1 Processor -- start 2024-03-22T19:13:06.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe724c20 0x55cdbe72ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe725580 0x55cdbe72ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe735fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe612d80 con 0x55cdbe724c20 2024-03-22T19:13:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe612a20 con 0x55cdbe725580 2024-03-22T19:13:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe4fa6a0 con 0x55cdbe726420 2024-03-22T19:13:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.834+0000 7f645d702b80 10 client.0 _ll_register_callbacks cb 0x55cdbe47e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe735fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6442000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe725580 0x55cdbe72ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe735fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47208/0 (socket says 172.21.15.86:47208) 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 -- 172.21.15.86:0/1137830540 learned_addr learned my addr 172.21.15.86:0/1137830540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6442a00700 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe724c20 0x55cdbe72ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 -- 172.21.15.86:0/1137830540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe725580 msgr2=0x55cdbe72ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe725580 0x55cdbe72ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 -- 172.21.15.86:0/1137830540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe724c20 msgr2=0x55cdbe72ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe724c20 0x55cdbe72ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.835+0000 7f6443400700 1 -- 172.21.15.86:0/1137830540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644410e1b0 con 0x55cdbe726420 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.836+0000 7f6443400700 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe735fa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f643c016e20 tx=0x7f643c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7498fb31fcff288 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.836+0000 7f642cc00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.836+0000 7f642cc00700 1 -- 172.21.15.86:0/1137830540 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c015bd0 con 0x55cdbe726420 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.837+0000 7f642cc00700 1 -- 172.21.15.86:0/1137830540 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f643c015d30 con 0x55cdbe726420 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.837+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdbe717ad0 con 0x55cdbe726420 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.837+0000 7f642cc00700 1 -- 172.21.15.86:0/1137830540 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f643c021400 con 0x55cdbe726420 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.837+0000 7f645d702b80 10 client.25888 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.838+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdbe4a4b60 con 0x55cdbe726420 2024-03-22T19:13:06.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.838+0000 7f642cc00700 1 -- 172.21.15.86:0/1137830540 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cdbe717ad0 con 0x55cdbe726420 2024-03-22T19:13:06.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f642cc00700 1 -- 172.21.15.86:0/1137830540 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f643c029070 con 0x55cdbe726420 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f645d702b80 10 client.25888 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f645d702b80 10 client.25888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f645d702b80 20 client.25888 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f645d702b80 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe73a800 0x55cdbe73cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f645d702b80 4 client.25888 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdbe66f700 con 0x55cdbe73a800 2024-03-22T19:13:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.839+0000 7f6442a00700 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe73a800 0x55cdbe73cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.840+0000 7f6442a00700 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe73a800 0x55cdbe73cc20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16199c037e3d3bd5 server_cookie=72539d5ed5b4d7fa in_seq=0 out_seq=0 2024-03-22T19:13:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.840+0000 7f642cc00700 10 client.25888 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.841+0000 7f642cc00700 1 -- 172.21.15.86:0/1137830540 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cdbe66f700 con 0x55cdbe73a800 2024-03-22T19:13:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.841+0000 7f642cc00700 10 client.25888 handle_command_reply: tid=0 2024-03-22T19:13:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.841+0000 7f645d702b80 1 client.25888 shutdown 2024-03-22T19:13:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.841+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 msgr2=0x55cdbe735fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.841+0000 7f645d702b80 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe735fa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f643c016e20 tx=0x7f643c004ad0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 shutdown_connections 2024-03-22T19:13:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f645d702b80 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe73a800 0x55cdbe73cc20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f645d702b80 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe735fa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f645d702b80 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe724c20 0x55cdbe72ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f645d702b80 1 --2- 172.21.15.86:0/1137830540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe725580 0x55cdbe72ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 >> 172.21.15.86:0/1137830540 conn(0x55cdbe69a220 msgr2=0x55cdbe71f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.842+0000 7f642cc00700 0 client.25888 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.843+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 shutdown_connections 2024-03-22T19:13:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.843+0000 7f645d702b80 1 -- 172.21.15.86:0/1137830540 wait complete. 2024-03-22T19:13:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.845+0000 7f645d702b80 20 client.25888 trim_cache size 0 max 16384 2024-03-22T19:13:06.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.890+0000 7f645d702b80 1 Processor -- start 2024-03-22T19:13:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.891+0000 7f645d702b80 1 -- start start 2024-03-22T19:13:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.892+0000 7f645d702b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.892+0000 7f645d702b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe735f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.892+0000 7f645d702b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe726800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.892+0000 7f645d702b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe72c5c0 con 0x55cdbe726420 2024-03-22T19:13:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.892+0000 7f645d702b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe736a10 con 0x55cdbe724c20 2024-03-22T19:13:06.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.892+0000 7f645d702b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe729eb0 con 0x55cdbe725580 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.893+0000 7f6442a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.893+0000 7f6442a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48034/0 (socket says 172.21.15.86:48034) 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.893+0000 7f6442a00700 1 -- 172.21.15.86:0/529175863 learned_addr learned my addr 172.21.15.86:0/529175863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.893+0000 7f6442000700 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe735f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.893+0000 7f6443400700 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe726800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6442000700 1 -- 172.21.15.86:0/529175863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 msgr2=0x55cdbe725000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6442000700 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6442000700 1 -- 172.21.15.86:0/529175863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 msgr2=0x55cdbe726800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6442000700 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe726800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6442000700 1 -- 172.21.15.86:0/529175863 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbe63cea0 con 0x55cdbe725580 2024-03-22T19:13:06.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6442000700 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe735f60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4b20340934a9fa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6438c00700 1 -- 172.21.15.86:0/529175863 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000b9d0 con 0x55cdbe725580 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6438c00700 1 -- 172.21.15.86:0/529175863 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6430003e20 con 0x55cdbe725580 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.894+0000 7f6438c00700 1 -- 172.21.15.86:0/529175863 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000f6c0 con 0x55cdbe725580 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 -- 172.21.15.86:0/529175863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 msgr2=0x55cdbe735f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe735f60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430005a90 comp rx=0 tx=0).stop 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 -- 172.21.15.86:0/529175863 shutdown_connections 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe726420 0x55cdbe726800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe724c20 0x55cdbe725000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 --2- 172.21.15.86:0/529175863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe725580 0x55cdbe735f60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.895+0000 7f645d702b80 1 -- 172.21.15.86:0/529175863 >> 172.21.15.86:0/529175863 conn(0x55cdbe69a220 msgr2=0x55cdbe72dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.904+0000 7f645d702b80 1 -- 172.21.15.86:0/529175863 shutdown_connections 2024-03-22T19:13:06.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.905+0000 7f645d702b80 1 -- 172.21.15.86:0/529175863 wait complete. 2024-03-22T19:13:06.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.905+0000 7f645d702b80 1 -- start start 2024-03-22T19:13:06.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.907+0000 7f645d702b80 1 Processor -- start 2024-03-22T19:13:06.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe725580 0x55cdbe7317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe731d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 0x55cdbe735050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe735590 con 0x55cdbe725580 2024-03-22T19:13:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe735700 con 0x55cdbe734bd0 2024-03-22T19:13:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbe7358c0 con 0x55cdbe726420 2024-03-22T19:13:06.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.910+0000 7f645d702b80 10 client.0 _ll_register_callbacks cb 0x7f643c0167c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:06.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.912+0000 7f6443400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 0x55cdbe735050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.912+0000 7f6443400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 0x55cdbe735050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48040/0 (socket says 172.21.15.86:48040) 2024-03-22T19:13:06.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.912+0000 7f6443400700 1 -- 172.21.15.86:0/4088586694 learned_addr learned my addr 172.21.15.86:0/4088586694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:06.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.912+0000 7f6442000700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe731d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.912+0000 7f6442a00700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe725580 0x55cdbe7317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.913+0000 7f6443400700 1 -- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 msgr2=0x55cdbe731d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.913+0000 7f6443400700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe731d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.913+0000 7f6443400700 1 -- 172.21.15.86:0/4088586694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe725580 msgr2=0x55cdbe7317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:06.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.913+0000 7f6443400700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe725580 0x55cdbe7317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.913+0000 7f6443400700 1 -- 172.21.15.86:0/4088586694 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdbe4f83b0 con 0x55cdbe734bd0 2024-03-22T19:13:06.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.914+0000 7f6443400700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 0x55cdbe735050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f643c015a20 tx=0x7f643c029700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d853baac60bda9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:06.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.914+0000 7f642cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:06.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.914+0000 7f642cc00700 1 -- 172.21.15.86:0/4088586694 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c021400 con 0x55cdbe734bd0 2024-03-22T19:13:06.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.914+0000 7f642cc00700 1 -- 172.21.15.86:0/4088586694 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f643c014750 con 0x55cdbe734bd0 2024-03-22T19:13:06.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.915+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f643001e040 con 0x55cdbe734bd0 2024-03-22T19:13:06.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.915+0000 7f642cc00700 1 -- 172.21.15.86:0/4088586694 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f643c0099b0 con 0x55cdbe734bd0 2024-03-22T19:13:06.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.915+0000 7f645d702b80 10 client.25895 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:06.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.915+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdbe4fccd0 con 0x55cdbe734bd0 2024-03-22T19:13:06.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f642cc00700 1 -- 172.21.15.86:0/4088586694 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643c038030 con 0x55cdbe734bd0 2024-03-22T19:13:06.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f642cc00700 1 -- 172.21.15.86:0/4088586694 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f643c029070 con 0x55cdbe734bd0 2024-03-22T19:13:06.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f645d702b80 10 client.25895 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:06.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f645d702b80 10 client.25895 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:13:06.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f645d702b80 20 client.25895 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:06.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f645d702b80 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe78d0b0 0x55cdbe78f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:06.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f645d702b80 4 client.25895 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:06.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.916+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cdbe4c5cf0 con 0x55cdbe78d0b0 2024-03-22T19:13:06.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.918+0000 7f6442a00700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe78d0b0 0x55cdbe78f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:06.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.918+0000 7f6442a00700 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe78d0b0 0x55cdbe78f4d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fcc14393f57b7cd server_cookie=3db32d9b06224264 in_seq=0 out_seq=0 2024-03-22T19:13:06.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.918+0000 7f642cc00700 10 client.25895 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:06.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.919+0000 7f642cc00700 1 -- 172.21.15.86:0/4088586694 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cdbe4c5cf0 con 0x55cdbe78d0b0 2024-03-22T19:13:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.919+0000 7f642cc00700 10 client.25895 handle_command_reply: tid=0 2024-03-22T19:13:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.919+0000 7f645d702b80 1 client.25895 shutdown 2024-03-22T19:13:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.920+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 msgr2=0x55cdbe735050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.920+0000 7f645d702b80 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 0x55cdbe735050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f643c015a20 tx=0x7f643c029700 comp rx=0 tx=0).stop 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.921+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 shutdown_connections 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.921+0000 7f645d702b80 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55cdbe78d0b0 0x55cdbe78f4d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.921+0000 7f642cc00700 0 client.25895 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.921+0000 7f645d702b80 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cdbe734bd0 0x55cdbe735050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.921+0000 7f645d702b80 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbe725580 0x55cdbe7317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.921+0000 7f645d702b80 1 --2- 172.21.15.86:0/4088586694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbe726420 0x55cdbe731d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.922+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 >> 172.21.15.86:0/4088586694 conn(0x55cdbe69a220 msgr2=0x55cdbe72c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.922+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 shutdown_connections 2024-03-22T19:13:06.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.922+0000 7f645d702b80 1 -- 172.21.15.86:0/4088586694 wait complete. 2024-03-22T19:13:06.925 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:06.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.923+0000 7f645d702b80 20 client.25895 trim_cache size 0 max 16384 2024-03-22T19:13:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.924+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64280a0c90 msgr2=0x7f64280a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.924+0000 7f6449c00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64280a0c90 0x7f64280a3150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f643c009ca0 tx=0x7f643c009040 comp rx=0 tx=0).stop 2024-03-22T19:13:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.924+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 msgr2=0x7f64441a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.924+0000 7f6449c00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f64441a5a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f643400cce0 tx=0x7f64340074a0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.926+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 shutdown_connections 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.926+0000 7f6449c00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64280a0c90 0x7f64280a3150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.926+0000 7f6449c00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6444108ee0 0x7f64441acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.926+0000 7f6449c00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64440721e0 0x7f64441a5a20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.926+0000 7f6449c00700 1 --2- 172.21.15.86:0/4059604399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6444108520 0x7f64441a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.926+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 >> 172.21.15.86:0/4059604399 conn(0x7f644406dd20 msgr2=0x7f644406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.927+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 shutdown_connections 2024-03-22T19:13:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:06.927+0000 7f6449c00700 1 -- 172.21.15.86:0/4059604399 wait complete. 2024-03-22T19:13:06.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:13:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.302+0000 7fccf5400700 1 -- 172.21.15.86:0/331184259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0100d40 msgr2=0x7fccf0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.302+0000 7fccf5400700 1 --2- 172.21.15.86:0/331184259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0100d40 0x7fccf0101160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcce00099c0 tx=0x7fcce0009cd0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 -- 172.21.15.86:0/331184259 shutdown_connections 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 --2- 172.21.15.86:0/331184259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccf01028d0 0x7fccf0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 --2- 172.21.15.86:0/331184259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0100d40 0x7fccf0101160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 --2- 172.21.15.86:0/331184259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf0101f30 0x7fccf0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 -- 172.21.15.86:0/331184259 >> 172.21.15.86:0/331184259 conn(0x7fccf00fc2b0 msgr2=0x7fccf00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 -- 172.21.15.86:0/331184259 shutdown_connections 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.303+0000 7fccf5400700 1 -- 172.21.15.86:0/331184259 wait complete. 2024-03-22T19:13:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.304+0000 7fccf5400700 1 Processor -- start 2024-03-22T19:13:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.304+0000 7fccf5400700 1 -- start start 2024-03-22T19:13:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccf5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf0100d40 0x7fccf019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccf5400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccf0101f30 0x7fccf019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccf5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 0x7fccf01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccf5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf01a4e50 con 0x7fccf0100d40 2024-03-22T19:13:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccf5400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf01a4f90 con 0x7fccf01028d0 2024-03-22T19:13:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccf5400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf01a5120 con 0x7fccf0101f30 2024-03-22T19:13:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccef200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 0x7fccf01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccef200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 0x7fccf01a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48046/0 (socket says 172.21.15.86:48046) 2024-03-22T19:13:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.305+0000 7fccef200700 1 -- 172.21.15.86:0/3318340947 learned_addr learned my addr 172.21.15.86:0/3318340947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccede00700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccf0101f30 0x7fccf019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccef200700 1 -- 172.21.15.86:0/3318340947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccf0101f30 msgr2=0x7fccf019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccef200700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccf0101f30 0x7fccf019d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccef200700 1 -- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf0100d40 msgr2=0x7fccf019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccef200700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf0100d40 0x7fccf019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccef200700 1 -- 172.21.15.86:0/3318340947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00095f0 con 0x7fccf01028d0 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fccef200700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 0x7fccf01a4880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007ee0 tx=0x7fcce800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.306+0000 7fcce7400700 1 -- 172.21.15.86:0/3318340947 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8017070 con 0x7fccf01028d0 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.307+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf01a5400 con 0x7fccf01028d0 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.307+0000 7fcce7400700 1 -- 172.21.15.86:0/3318340947 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce800f040 con 0x7fccf01028d0 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.307+0000 7fcce7400700 1 -- 172.21.15.86:0/3318340947 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8005740 con 0x7fccf01028d0 2024-03-22T19:13:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.307+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf01a5910 con 0x7fccf01028d0 2024-03-22T19:13:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.311+0000 7fcce7400700 1 -- 172.21.15.86:0/3318340947 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fcce80058a0 con 0x7fccf01028d0 2024-03-22T19:13:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.314+0000 7fcd08ee5b80 1 Processor -- start 2024-03-22T19:13:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.314+0000 7fcce7400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fccd40a0a90 0x7fccd40a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.315+0000 7fcd08ee5b80 1 -- start start 2024-03-22T19:13:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.315+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c560040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.316+0000 7fcce7400700 1 -- 172.21.15.86:0/3318340947 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fcce80cb8a0 con 0x7fccf01028d0 2024-03-22T19:13:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.316+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c5618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c4b8990 con 0x555e9c55fc20 2024-03-22T19:13:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c4b6350 con 0x555e9c560580 2024-03-22T19:13:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c4b1c40 con 0x555e9c561420 2024-03-22T19:13:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fccede00700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fccd40a0a90 0x7fccd40a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fccee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c560040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fccee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c560040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38586/0 (socket says 172.21.15.86:38586) 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.317+0000 7fccee800700 1 -- 172.21.15.86:0/3703959056 learned_addr learned my addr 172.21.15.86:0/3703959056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccef200700 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c5618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccee800700 1 -- 172.21.15.86:0/3703959056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 msgr2=0x555e9c5618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccee800700 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c5618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccee800700 1 -- 172.21.15.86:0/3703959056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 msgr2=0x555e9c560e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccee800700 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccee800700 1 -- 172.21.15.86:0/3703959056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e9c3333b0 con 0x555e9c55fc20 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccef200700 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c5618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccede00700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fccd40a0a90 0x7fccd40a2f50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fccdc005e20 tx=0x7fccdc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.318+0000 7fccee800700 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c560040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006010 tx=0x7fcce0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d282d785d29ec server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.319+0000 7fccdb400700 1 -- 172.21.15.86:0/3703959056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce001d070 con 0x555e9c55fc20 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.319+0000 7fccdb400700 1 -- 172.21.15.86:0/3703959056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce0004500 con 0x555e9c55fc20 2024-03-22T19:13:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.319+0000 7fccdb400700 1 -- 172.21.15.86:0/3703959056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0003de0 con 0x555e9c55fc20 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3703959056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 msgr2=0x555e9c560040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c560040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006010 tx=0x7fcce0009530 comp rx=0 tx=0).stop 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3703959056 shutdown_connections 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c5618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c560040 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3703959056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3703959056 >> 172.21.15.86:0/3703959056 conn(0x555e9c4d5220 msgr2=0x555e9c55abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.320+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3703959056 shutdown_connections 2024-03-22T19:13:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.321+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3703959056 wait complete. 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.321+0000 7fcd08ee5b80 1 -- start start 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 Processor -- start 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c569aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c560580 0x555e9c569fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c561420 0x555e9c571020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c44dd80 con 0x555e9c55fc20 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c44da20 con 0x555e9c561420 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c3356a0 con 0x555e9c560580 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.322+0000 7fcd08ee5b80 10 client.0 _ll_register_callbacks cb 0x555e9c2b93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.323+0000 7fccee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c569aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.323+0000 7fccee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c569aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38592/0 (socket says 172.21.15.86:38592) 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.323+0000 7fccee800700 1 -- 172.21.15.86:0/3948851459 learned_addr learned my addr 172.21.15.86:0/3948851459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.323+0000 7fccef200700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c561420 0x555e9c571020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.323+0000 7fccede00700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c560580 0x555e9c569fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.324+0000 7fccee800700 1 -- 172.21.15.86:0/3948851459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c560580 msgr2=0x555e9c569fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.324+0000 7fccee800700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c560580 0x555e9c569fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.324+0000 7fccee800700 1 -- 172.21.15.86:0/3948851459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c561420 msgr2=0x555e9c571020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.324+0000 7fccee800700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c561420 0x555e9c571020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.324+0000 7fccee800700 1 -- 172.21.15.86:0/3948851459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf0109ad0 con 0x555e9c55fc20 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.324+0000 7fccede00700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c560580 0x555e9c569fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.325+0000 7fccee800700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c569aa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcce0005fe0 tx=0x7fcce000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2193aef049fb57 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.325+0000 7fcccfe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.326+0000 7fcccfe00700 1 -- 172.21.15.86:0/3948851459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce001d050 con 0x555e9c55fc20 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.326+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e9c552ad0 con 0x555e9c55fc20 2024-03-22T19:13:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.326+0000 7fcccfe00700 1 -- 172.21.15.86:0/3948851459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fcce0003fb0 con 0x555e9c55fc20 2024-03-22T19:13:07.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.326+0000 7fcccfe00700 1 -- 172.21.15.86:0/3948851459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcce0004170 con 0x555e9c55fc20 2024-03-22T19:13:07.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.326+0000 7fcd08ee5b80 10 client.15858 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:07.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.326+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e9c2dfb60 con 0x555e9c55fc20 2024-03-22T19:13:07.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcccfe00700 1 -- 172.21.15.86:0/3948851459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e9c552ad0 con 0x555e9c55fc20 2024-03-22T19:13:07.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcccfe00700 1 -- 172.21.15.86:0/3948851459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcce0029030 con 0x555e9c55fc20 2024-03-22T19:13:07.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcd08ee5b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:07.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcd08ee5b80 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcd08ee5b80 20 client.15858 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5751c0 0x555e9c5775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcd08ee5b80 4 client.15858 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e9c4aa700 con 0x555e9c5751c0 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.327+0000 7fccede00700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5751c0 0x555e9c5775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.328+0000 7fccede00700 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5751c0 0x555e9c5775b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=266950a2feb7701 server_cookie=7e64408b5603d4b7 in_seq=0 out_seq=0 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.328+0000 7fcccfe00700 10 client.15858 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcccfe00700 1 -- 172.21.15.86:0/3948851459 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555e9c4aa700 con 0x555e9c5751c0 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcccfe00700 10 client.15858 handle_command_reply: tid=0 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 client.15858 shutdown 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 msgr2=0x555e9c569aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c569aa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcce0005fe0 tx=0x7fcce000bcf0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 shutdown_connections 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5751c0 0x555e9c5775b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c561420 0x555e9c571020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c55fc20 0x555e9c569aa0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/3948851459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c560580 0x555e9c569fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcccfe00700 0 client.15858 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.329+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 >> 172.21.15.86:0/3948851459 conn(0x555e9c4d5220 msgr2=0x555e9c55a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.330+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 shutdown_connections 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.330+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/3948851459 wait complete. 2024-03-22T19:13:07.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.330+0000 7fcd08ee5b80 20 client.15858 trim_cache size 0 max 16384 2024-03-22T19:13:07.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.343+0000 7fcd08ee5b80 1 Processor -- start 2024-03-22T19:13:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 -- start start 2024-03-22T19:13:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c56efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c567960 0x555e9c568740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c555210 con 0x555e9c567960 2024-03-22T19:13:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c554f80 con 0x555e9c560580 2024-03-22T19:13:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c56ac90 con 0x555e9c561420 2024-03-22T19:13:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccede00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c56efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccede00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c56efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47258/0 (socket says 172.21.15.86:47258) 2024-03-22T19:13:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccede00700 1 -- 172.21.15.86:0/4129115615 learned_addr learned my addr 172.21.15.86:0/4129115615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccef200700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c567960 0x555e9c568740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccee800700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccede00700 1 -- 172.21.15.86:0/4129115615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 msgr2=0x555e9c560000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.344+0000 7fccede00700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccede00700 1 -- 172.21.15.86:0/4129115615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c567960 msgr2=0x555e9c568740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccede00700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c567960 0x555e9c568740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccede00700 1 -- 172.21.15.86:0/4129115615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e9c477ea0 con 0x555e9c561420 2024-03-22T19:13:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccee800700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccef200700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c567960 0x555e9c568740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccede00700 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c56efd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00ef50 tx=0x7fccdc011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe532bd0137a32c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccdbe00700 1 -- 172.21.15.86:0/4129115615 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc024070 con 0x555e9c561420 2024-03-22T19:13:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccdbe00700 1 -- 172.21.15.86:0/4129115615 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccdc028070 con 0x555e9c561420 2024-03-22T19:13:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.345+0000 7fccdbe00700 1 -- 172.21.15.86:0/4129115615 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc005280 con 0x555e9c561420 2024-03-22T19:13:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/4129115615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 msgr2=0x555e9c56efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c56efd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00ef50 tx=0x7fccdc011000 comp rx=0 tx=0).stop 2024-03-22T19:13:07.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/4129115615 shutdown_connections 2024-03-22T19:13:07.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c567960 0x555e9c568740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c560000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/4129115615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c561420 0x555e9c56efd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/4129115615 >> 172.21.15.86:0/4129115615 conn(0x555e9c4d5220 msgr2=0x555e9c568fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/4129115615 shutdown_connections 2024-03-22T19:13:07.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/4129115615 wait complete. 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.346+0000 7fcd08ee5b80 1 -- start start 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 Processor -- start 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c56e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 0x555e9c56ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c567960 0x555e9c56cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c56d5e0 con 0x555e9c561420 2024-03-22T19:13:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c56d750 con 0x555e9c560580 2024-03-22T19:13:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9c56d910 con 0x555e9c567960 2024-03-22T19:13:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fcd08ee5b80 10 client.0 _ll_register_callbacks cb 0x7fcce0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fccede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 0x555e9c56ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fccede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 0x555e9c56ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38610/0 (socket says 172.21.15.86:38610) 2024-03-22T19:13:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fccede00700 1 -- 172.21.15.86:0/1353336660 learned_addr learned my addr 172.21.15.86:0/1353336660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fccee800700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c56e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.347+0000 7fccef200700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c567960 0x555e9c56cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccede00700 1 -- 172.21.15.86:0/1353336660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c567960 msgr2=0x555e9c56cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccede00700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c567960 0x555e9c56cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccede00700 1 -- 172.21.15.86:0/1353336660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 msgr2=0x555e9c56e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccede00700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c56e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccede00700 1 -- 172.21.15.86:0/1353336660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc00ec80 con 0x555e9c561420 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccef200700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c567960 0x555e9c56cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fccede00700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 0x555e9c56ee30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fccdc024380 tx=0x7fccdc00a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab024b5b1b3c15e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fcccfe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fcccfe00700 1 -- 172.21.15.86:0/1353336660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc024050 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fcccfe00700 1 -- 172.21.15.86:0/1353336660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fccdc004910 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.348+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcce0004500 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcccfe00700 1 -- 172.21.15.86:0/1353336660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccdc004ad0 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 10 client.15864 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e9c337cd0 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcccfe00700 1 -- 172.21.15.86:0/1353336660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce0004500 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcccfe00700 1 -- 172.21.15.86:0/1353336660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fccdc0377d0 con 0x555e9c561420 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 10 client.15864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 20 client.15864 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5c8000 0x555e9c5ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 4 client.15864 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e9c300cf0 con 0x555e9c5c8000 2024-03-22T19:13:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.349+0000 7fccee800700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5c8000 0x555e9c5ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcccfe00700 10 client.15864 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fccee800700 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5c8000 0x555e9c5ca420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41e461ee6a2262fb server_cookie=835caec6663c2985 in_seq=0 out_seq=0 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcccfe00700 1 -- 172.21.15.86:0/1353336660 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e9c300cf0 con 0x555e9c5c8000 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcccfe00700 10 client.15864 handle_command_reply: tid=0 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcd08ee5b80 1 client.15864 shutdown 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 msgr2=0x555e9c56ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 0x555e9c56ee30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fccdc024380 tx=0x7fccdc00a930 comp rx=0 tx=0).stop 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 shutdown_connections 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.350+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x555e9c5c8000 0x555e9c5ca420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e9c567960 0x555e9c56cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e9c560580 0x555e9c56e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 1 --2- 172.21.15.86:0/1353336660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e9c561420 0x555e9c56ee30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcccfe00700 0 client.15864 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 >> 172.21.15.86:0/1353336660 conn(0x555e9c4d5220 msgr2=0x555e9c568740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 shutdown_connections 2024-03-22T19:13:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 1 -- 172.21.15.86:0/1353336660 wait complete. 2024-03-22T19:13:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fcd08ee5b80 20 client.15864 trim_cache size 0 max 16384 2024-03-22T19:13:07.365 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fccd40a0a90 msgr2=0x7fccd40a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fccf5400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fccd40a0a90 0x7fccd40a2f50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fccdc005e20 tx=0x7fccdc005ed0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 msgr2=0x7fccf01a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.351+0000 7fccf5400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 0x7fccf01a4880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007ee0 tx=0x7fcce800d3b0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 shutdown_connections 2024-03-22T19:13:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fccd40a0a90 0x7fccd40a2f50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf01028d0 0x7fccf01a4880 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf0100d40 0x7fccf019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 --2- 172.21.15.86:0/3318340947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccf0101f30 0x7fccf019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 >> 172.21.15.86:0/3318340947 conn(0x7fccf00fc2b0 msgr2=0x7fccf0107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 shutdown_connections 2024-03-22T19:13:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.352+0000 7fccf5400700 1 -- 172.21.15.86:0/3318340947 wait complete. 2024-03-22T19:13:07.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:13:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:07 smithi167 ceph-mon[47728]: pgmap v350: 97 pgs: 97 active+clean; 5.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 270 MiB/s wr, 107 op/s 2024-03-22T19:13:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.691+0000 7f189aa00700 1 -- 172.21.15.86:0/3005465095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894103fa0 msgr2=0x7f1894104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.691+0000 7f189aa00700 1 --2- 172.21.15.86:0/3005465095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894103fa0 0x7f1894104400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1880009990 tx=0x7f1880009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.693+0000 7f189aa00700 1 -- 172.21.15.86:0/3005465095 shutdown_connections 2024-03-22T19:13:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.693+0000 7f189aa00700 1 --2- 172.21.15.86:0/3005465095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894104940 0x7f189410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.693+0000 7f189aa00700 1 --2- 172.21.15.86:0/3005465095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1894102db0 0x7f18941031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.694+0000 7f189aa00700 1 --2- 172.21.15.86:0/3005465095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894103fa0 0x7f1894104400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.694+0000 7f189aa00700 1 -- 172.21.15.86:0/3005465095 >> 172.21.15.86:0/3005465095 conn(0x7f18940fe320 msgr2=0x7f1894100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.696+0000 7f189aa00700 1 -- 172.21.15.86:0/3005465095 shutdown_connections 2024-03-22T19:13:07.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.696+0000 7f189aa00700 1 -- 172.21.15.86:0/3005465095 wait complete. 2024-03-22T19:13:07.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.701+0000 7f189aa00700 1 Processor -- start 2024-03-22T19:13:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.701+0000 7f189aa00700 1 -- start start 2024-03-22T19:13:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.704+0000 7f189aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1894102db0 0x7f189406f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.704+0000 7f189aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 0x7f189406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.707+0000 7f189aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 0x7f18941a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.707+0000 7f189aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894072c50 con 0x7f1894102db0 2024-03-22T19:13:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.707+0000 7f189aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894072dc0 con 0x7f1894103fa0 2024-03-22T19:13:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.707+0000 7f189aa00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941a94d0 con 0x7f1894104940 2024-03-22T19:13:07.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.708+0000 7f189a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 0x7f18941a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.708+0000 7f189a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 0x7f18941a8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47306/0 (socket says 172.21.15.86:47306) 2024-03-22T19:13:07.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.708+0000 7f1898c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 0x7f189406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.708+0000 7f1898c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 0x7f189406fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48090/0 (socket says 172.21.15.86:48090) 2024-03-22T19:13:07.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f1898c00700 1 -- 172.21.15.86:0/781074157 learned_addr learned my addr 172.21.15.86:0/781074157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f1898c00700 1 -- 172.21.15.86:0/781074157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 msgr2=0x7f18941a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f1898c00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 0x7f18941a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f1898c00700 1 -- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1894102db0 msgr2=0x7f189406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f1898c00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1894102db0 0x7f189406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f1898c00700 1 -- 172.21.15.86:0/781074157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c009c40 con 0x7f1894103fa0 2024-03-22T19:13:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.709+0000 7f189a000700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 0x7f18941a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:07.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.710+0000 7f1898c00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 0x7f189406fbc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1880000c00 tx=0x7f18800057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.710+0000 7f1892000700 1 -- 172.21.15.86:0/781074157 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188001d070 con 0x7f1894103fa0 2024-03-22T19:13:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.711+0000 7f1892000700 1 -- 172.21.15.86:0/781074157 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188000fc00 con 0x7f1894103fa0 2024-03-22T19:13:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.711+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18800095f0 con 0x7f1894103fa0 2024-03-22T19:13:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.711+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941a9960 con 0x7f1894103fa0 2024-03-22T19:13:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.711+0000 7f1892000700 1 -- 172.21.15.86:0/781074157 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880017650 con 0x7f1894103fa0 2024-03-22T19:13:07.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:07 smithi131 ceph-mon[78559]: pgmap v350: 97 pgs: 97 active+clean; 5.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 270 MiB/s wr, 107 op/s 2024-03-22T19:13:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.719+0000 7f1892000700 1 -- 172.21.15.86:0/781074157 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f18800177b0 con 0x7f1894103fa0 2024-03-22T19:13:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.720+0000 7f18afaf4b80 1 Processor -- start 2024-03-22T19:13:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.720+0000 7f18afaf4b80 1 -- start start 2024-03-22T19:13:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.721+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.721+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af353f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.721+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3546270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.721+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af3496990 con 0x563af353dde0 2024-03-22T19:13:07.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.721+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af3494350 con 0x563af353f020 2024-03-22T19:13:07.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.721+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af348fc40 con 0x563af353f9e0 2024-03-22T19:13:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.722+0000 7f1892000700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f187c0a0be0 0x7f187c0a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.722+0000 7f1892000700 1 -- 172.21.15.86:0/781074157 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f18800ce460 con 0x7f1894103fa0 2024-03-22T19:13:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.722+0000 7f189a000700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f187c0a0be0 0x7f187c0a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.722+0000 7f189a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38642/0 (socket says 172.21.15.86:38642) 2024-03-22T19:13:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 -- 172.21.15.86:0/3148878583 learned_addr learned my addr 172.21.15.86:0/3148878583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f1899600700 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af353f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f1898c00700 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3546270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 -- 172.21.15.86:0/3148878583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 msgr2=0x563af3546270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3546270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 -- 172.21.15.86:0/3148878583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 msgr2=0x563af353f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af353f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.723+0000 7f189a000700 1 -- 172.21.15.86:0/3148878583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563af33113b0 con 0x563af353dde0 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.725+0000 7f1899600700 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af353f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.725+0000 7f189a000700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f187c0a0be0 0x7f187c0a30a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f188c010b40 tx=0x7f188c010f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.725+0000 7f189a000700 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353e220 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f188c0176e0 tx=0x7f188c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54033b30703a4a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.726+0000 7f1886600700 1 -- 172.21.15.86:0/3148878583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00e040 con 0x563af353dde0 2024-03-22T19:13:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.726+0000 7f1886600700 1 -- 172.21.15.86:0/3148878583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188c019880 con 0x563af353dde0 2024-03-22T19:13:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.726+0000 7f1886600700 1 -- 172.21.15.86:0/3148878583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c004280 con 0x563af353dde0 2024-03-22T19:13:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.727+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3148878583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 msgr2=0x563af353e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.727+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353e220 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f188c0176e0 tx=0x7f188c00b040 comp rx=0 tx=0).stop 2024-03-22T19:13:07.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.729+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3148878583 shutdown_connections 2024-03-22T19:13:07.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.729+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3546270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.729+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353e220 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.729+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3148878583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af353f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.729+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3148878583 >> 172.21.15.86:0/3148878583 conn(0x563af34b3220 msgr2=0x563af3538e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.730+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3148878583 shutdown_connections 2024-03-22T19:13:07.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.730+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3148878583 wait complete. 2024-03-22T19:13:07.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.731+0000 7f18afaf4b80 1 -- start start 2024-03-22T19:13:07.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.731+0000 7f18afaf4b80 1 Processor -- start 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353dde0 0x563af3550490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f020 0x563af35509d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 0x563af3557a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af342bd80 con 0x563af353f020 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af342ba20 con 0x563af353f9e0 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af33136a0 con 0x563af353dde0 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.732+0000 7f18afaf4b80 10 client.0 _ll_register_callbacks cb 0x563af32973a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.733+0000 7f1898c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f020 0x563af35509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.733+0000 7f1899600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353dde0 0x563af3550490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.733+0000 7f189a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 0x563af3557a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.733+0000 7f189a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 0x563af3557a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48098/0 (socket says 172.21.15.86:48098) 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.733+0000 7f189a000700 1 -- 172.21.15.86:0/4137502731 learned_addr learned my addr 172.21.15.86:0/4137502731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f189a000700 1 -- 172.21.15.86:0/4137502731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353dde0 msgr2=0x563af3550490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f189a000700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353dde0 0x563af3550490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f189a000700 1 -- 172.21.15.86:0/4137502731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f020 msgr2=0x563af35509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f189a000700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f020 0x563af35509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f189a000700 1 -- 172.21.15.86:0/4137502731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0179e0 con 0x563af353f9e0 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f1899600700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353dde0 0x563af3550490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.734+0000 7f1898c00700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f020 0x563af35509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.735+0000 7f189a000700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 0x563af3557a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f188c024040 tx=0x7f188c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e83752196f2aff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.735+0000 7f187aa00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.736+0000 7f187aa00700 1 -- 172.21.15.86:0/4137502731 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00e040 con 0x563af353f9e0 2024-03-22T19:13:07.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.736+0000 7f187aa00700 1 -- 172.21.15.86:0/4137502731 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f188c003c10 con 0x563af353f9e0 2024-03-22T19:13:07.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.736+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563af3530c30 con 0x563af353f9e0 2024-03-22T19:13:07.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.737+0000 7f187aa00700 1 -- 172.21.15.86:0/4137502731 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f188c007500 con 0x563af353f9e0 2024-03-22T19:13:07.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.737+0000 7f18afaf4b80 10 client.25922 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:07.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.737+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563af32bdb60 con 0x563af353f9e0 2024-03-22T19:13:07.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.737+0000 7f187aa00700 1 -- 172.21.15.86:0/4137502731 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563af3530c30 con 0x563af353f9e0 2024-03-22T19:13:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f187aa00700 1 -- 172.21.15.86:0/4137502731 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f188c02a040 con 0x563af353f9e0 2024-03-22T19:13:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f18afaf4b80 10 client.25922 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f18afaf4b80 10 client.25922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f18afaf4b80 20 client.25922 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:07.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af354bac0 0x563af354dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f18afaf4b80 4 client.25922 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:07.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.740+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563af3488700 con 0x563af354bac0 2024-03-22T19:13:07.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.743+0000 7f1899600700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af354bac0 0x563af354dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.743+0000 7f1899600700 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af354bac0 0x563af354dee0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc257cfd11cea7f6 server_cookie=d84874ca18b88ebb in_seq=0 out_seq=0 2024-03-22T19:13:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.743+0000 7f187aa00700 10 client.25922 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.744+0000 7f187aa00700 1 -- 172.21.15.86:0/4137502731 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563af3488700 con 0x563af354bac0 2024-03-22T19:13:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.744+0000 7f187aa00700 10 client.25922 handle_command_reply: tid=0 2024-03-22T19:13:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.744+0000 7f18afaf4b80 1 client.25922 shutdown 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 msgr2=0x563af3557a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 0x563af3557a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f188c024040 tx=0x7f188c000f40 comp rx=0 tx=0).stop 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 shutdown_connections 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af354bac0 0x563af354dee0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f9e0 0x563af3557a10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353dde0 0x563af3550490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/4137502731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f020 0x563af35509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 >> 172.21.15.86:0/4137502731 conn(0x563af34b3220 msgr2=0x563af3538350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f187aa00700 0 client.25922 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 shutdown_connections 2024-03-22T19:13:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.745+0000 7f18afaf4b80 1 -- 172.21.15.86:0/4137502731 wait complete. 2024-03-22T19:13:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.746+0000 7f18afaf4b80 20 client.25922 trim_cache size 0 max 16384 2024-03-22T19:13:07.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:07 smithi086 ceph-mon[38032]: pgmap v350: 97 pgs: 97 active+clean; 5.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 270 MiB/s wr, 107 op/s 2024-03-22T19:13:07.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.779+0000 7f18afaf4b80 1 Processor -- start 2024-03-22T19:13:07.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.779+0000 7f18afaf4b80 1 -- start start 2024-03-22T19:13:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af35579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3547190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af353a5c0 con 0x563af353dde0 2024-03-22T19:13:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af3535740 con 0x563af353f020 2024-03-22T19:13:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f18afaf4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af35484f0 con 0x563af353f9e0 2024-03-22T19:13:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f189a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3547190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f189a000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3547190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47330/0 (socket says 172.21.15.86:47330) 2024-03-22T19:13:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.780+0000 7f189a000700 1 -- 172.21.15.86:0/3259960984 learned_addr learned my addr 172.21.15.86:0/3259960984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1898c00700 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af35579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1899600700 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1898c00700 1 -- 172.21.15.86:0/3259960984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 msgr2=0x563af3547190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1898c00700 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3547190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1898c00700 1 -- 172.21.15.86:0/3259960984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 msgr2=0x563af353f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1898c00700 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1898c00700 1 -- 172.21.15.86:0/3259960984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563af3455ea0 con 0x563af353f020 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.781+0000 7f1899600700 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.782+0000 7f1898c00700 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af35579d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x563af353ede0 tx=0x7f188000ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3badf4a044779869 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.783+0000 7f1887000700 1 -- 172.21.15.86:0/3259960984 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188009c300 con 0x563af353f020 2024-03-22T19:13:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.784+0000 7f1887000700 1 -- 172.21.15.86:0/3259960984 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188009bc00 con 0x563af353f020 2024-03-22T19:13:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.784+0000 7f1887000700 1 -- 172.21.15.86:0/3259960984 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188009ab60 con 0x563af353f020 2024-03-22T19:13:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3259960984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 msgr2=0x563af35579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af35579d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x563af353ede0 tx=0x7f188000ff50 comp rx=0 tx=0).stop 2024-03-22T19:13:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3259960984 shutdown_connections 2024-03-22T19:13:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f9e0 0x563af3547190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353dde0 0x563af353f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/3259960984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353f020 0x563af35579d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3259960984 >> 172.21.15.86:0/3259960984 conn(0x563af34b3220 msgr2=0x563af3545fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3259960984 shutdown_connections 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.785+0000 7f18afaf4b80 1 -- 172.21.15.86:0/3259960984 wait complete. 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.786+0000 7f18afaf4b80 1 -- start start 2024-03-22T19:13:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.786+0000 7f18afaf4b80 1 Processor -- start 2024-03-22T19:13:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.787+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 0x563af3553ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.787+0000 7f18afaf4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 0x563af3554020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f1899600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 0x563af3553ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f1898c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 0x563af3554020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f1898c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 0x563af3554020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47334/0 (socket says 172.21.15.86:47334) 2024-03-22T19:13:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f1899600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 0x563af3553ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48124/0 (socket says 172.21.15.86:48124) 2024-03-22T19:13:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f1898c00700 1 -- 172.21.15.86:0/1722159806 learned_addr learned my addr 172.21.15.86:0/1722159806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f9e0 0x563af3556ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.788+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af3557530 con 0x563af353f9e0 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af35576a0 con 0x563af353dde0 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af3557860 con 0x563af353f020 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f18afaf4b80 10 client.0 _ll_register_callbacks cb 0x7f188c01c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f189a000700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f9e0 0x563af3556ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f1899600700 1 -- 172.21.15.86:0/1722159806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 msgr2=0x563af3554020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f1899600700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 0x563af3554020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f1899600700 1 -- 172.21.15.86:0/1722159806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f9e0 msgr2=0x563af3556ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f1899600700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f9e0 0x563af3556ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.789+0000 7f1899600700 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18800bb040 con 0x563af353dde0 2024-03-22T19:13:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f1898c00700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 0x563af3554020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f189a000700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f9e0 0x563af3556ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f1899600700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 0x563af3553ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x563af353edb0 tx=0x7f1888016760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af836d8efd29bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f187aa00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f187aa00700 1 -- 172.21.15.86:0/1722159806 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888022440 con 0x563af353dde0 2024-03-22T19:13:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f188c019880 con 0x563af353dde0 2024-03-22T19:13:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.790+0000 7f187aa00700 1 -- 172.21.15.86:0/1722159806 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f18880225a0 con 0x563af353dde0 2024-03-22T19:13:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f187aa00700 1 -- 172.21.15.86:0/1722159806 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1888022780 con 0x563af353dde0 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 10 client.25928 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563af3315cd0 con 0x563af353dde0 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f187aa00700 1 -- 172.21.15.86:0/1722159806 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1888022da0 con 0x563af353dde0 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f187aa00700 1 -- 172.21.15.86:0/1722159806 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f188802db20 con 0x563af353dde0 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 10 client.25928 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 20 client.25928 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af35b6fd0 0x563af35b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 4 client.25928 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.791+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f188c019880 con 0x563af35b6fd0 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.792+0000 7f1898c00700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af35b6fd0 0x563af35b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.792+0000 7f187aa00700 10 client.25928 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.792+0000 7f1898c00700 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af35b6fd0 0x563af35b93f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22d8ff391d89ef7 server_cookie=2b24c9fbcbfec4c0 in_seq=0 out_seq=0 2024-03-22T19:13:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.792+0000 7f187aa00700 1 -- 172.21.15.86:0/1722159806 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f188c019880 con 0x563af35b6fd0 2024-03-22T19:13:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.792+0000 7f187aa00700 10 client.25928 handle_command_reply: tid=0 2024-03-22T19:13:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.792+0000 7f18afaf4b80 1 client.25928 shutdown 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 msgr2=0x563af3553ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 0x563af3553ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x563af353edb0 tx=0x7f1888016760 comp rx=0 tx=0).stop 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 shutdown_connections 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563af35b6fd0 0x563af35b93f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563af353f9e0 0x563af3556ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563af353dde0 0x563af3553ae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 --2- 172.21.15.86:0/1722159806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563af353f020 0x563af3554020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 >> 172.21.15.86:0/1722159806 conn(0x563af34b3220 msgr2=0x563af354dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f187aa00700 0 client.25928 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 shutdown_connections 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.793+0000 7f18afaf4b80 1 -- 172.21.15.86:0/1722159806 wait complete. 2024-03-22T19:13:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.794+0000 7f18afaf4b80 20 client.25928 trim_cache size 0 max 16384 2024-03-22T19:13:07.797 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.794+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f187c0a0be0 msgr2=0x7f187c0a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.794+0000 7f189aa00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f187c0a0be0 0x7f187c0a30a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f188c010b40 tx=0x7f188c010f00 comp rx=0 tx=0).stop 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.794+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 msgr2=0x7f189406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.794+0000 7f189aa00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 0x7f189406fbc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1880000c00 tx=0x7f18800057f0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 shutdown_connections 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f187c0a0be0 0x7f187c0a30a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1894104940 0x7f18941a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1894102db0 0x7f189406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 --2- 172.21.15.86:0/781074157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1894103fa0 0x7f189406fbc0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1880000c00 tx=0x7f18800057f0 comp rx=0 tx=0).stop 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 >> 172.21.15.86:0/781074157 conn(0x7f18940fe320 msgr2=0x7f1894107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 shutdown_connections 2024-03-22T19:13:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:07.795+0000 7f189aa00700 1 -- 172.21.15.86:0/781074157 wait complete. 2024-03-22T19:13:08.827 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:13:08.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:13:09.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.164+0000 7fe903200700 1 -- 172.21.15.86:0/2419700277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 msgr2=0x7fe8f80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.165+0000 7fe903200700 1 --2- 172.21.15.86:0/2419700277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f80a6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009990 tx=0x7fe8ec009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.166+0000 7fe903200700 1 -- 172.21.15.86:0/2419700277 shutdown_connections 2024-03-22T19:13:09.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.166+0000 7fe903200700 1 --2- 172.21.15.86:0/2419700277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 0x7fe8f80ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.166+0000 7fe903200700 1 --2- 172.21.15.86:0/2419700277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f80a4c90 0x7fe8f80a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.166+0000 7fe903200700 1 --2- 172.21.15.86:0/2419700277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f80a6240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.166+0000 7fe903200700 1 -- 172.21.15.86:0/2419700277 >> 172.21.15.86:0/2419700277 conn(0x7fe8f80a0120 msgr2=0x7fe8f80a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.167+0000 7fe903200700 1 -- 172.21.15.86:0/2419700277 shutdown_connections 2024-03-22T19:13:09.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.167+0000 7fe903200700 1 -- 172.21.15.86:0/2419700277 wait complete. 2024-03-22T19:13:09.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.168+0000 7fe903200700 1 Processor -- start 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 -- start start 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f80a4c90 0x7fe8f8143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f8143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 0x7fe8f814a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f814af20 con 0x7fe8f80a4c90 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f814b060 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.169+0000 7fe903200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f814b1f0 con 0x7fe8f80a6780 2024-03-22T19:13:09.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.171+0000 7fe901400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f8143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.171+0000 7fe901400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f8143980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48412/0 (socket says 172.21.15.86:48412) 2024-03-22T19:13:09.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.171+0000 7fe901400700 1 -- 172.21.15.86:0/1947307050 learned_addr learned my addr 172.21.15.86:0/1947307050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.171+0000 7fe901e00700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f80a4c90 0x7fe8f8143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe902800700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 0x7fe8f814a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe901400700 1 -- 172.21.15.86:0/1947307050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 msgr2=0x7fe8f814a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe901400700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 0x7fe8f814a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe901400700 1 -- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f80a4c90 msgr2=0x7fe8f8143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe901400700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f80a4c90 0x7fe8f8143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe901400700 1 -- 172.21.15.86:0/1947307050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0095f0 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe902800700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 0x7fe8f814a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe901400700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f8143980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b5c0 tx=0x7fe8ec003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe8f6a00700 1 -- 172.21.15.86:0/1947307050 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec01d070 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe8f6a00700 1 -- 172.21.15.86:0/1947307050 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8ec00f460 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.172+0000 7fe8f6a00700 1 -- 172.21.15.86:0/1947307050 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec017650 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.173+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f814b470 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.174+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f814b7e0 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.180+0000 7fe9182f0b80 1 Processor -- start 2024-03-22T19:13:09.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.181+0000 7fe8f6a00700 1 -- 172.21.15.86:0/1947307050 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe8ec00f5d0 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.182+0000 7fe9182f0b80 1 -- start start 2024-03-22T19:13:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x5628069682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628069690a0 0x562806969520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806969a60 0x5628069702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628068c0a10 con 0x5628069690a0 2024-03-22T19:13:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628068be3d0 con 0x562806967e60 2024-03-22T19:13:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628068b9cc0 con 0x562806969a60 2024-03-22T19:13:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe8f6a00700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe8e40a0b60 0x7fe8e40a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.183+0000 7fe8f6a00700 1 -- 172.21.15.86:0/1947307050 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe8ec0cd030 con 0x7fe8f80a5dc0 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe901e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628069690a0 0x562806969520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe901400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806969a60 0x5628069702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe901400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806969a60 0x5628069702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:50098/0 (socket says 172.21.15.86:50098) 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe902800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x5628069682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe901400700 1 -- 172.21.15.86:0/1773316730 learned_addr learned my addr 172.21.15.86:0/1773316730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe902800700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe8e40a0b60 0x7fe8e40a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe902800700 1 -- 172.21.15.86:0/1773316730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806969a60 msgr2=0x5628069702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe902800700 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806969a60 0x5628069702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe902800700 1 -- 172.21.15.86:0/1773316730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628069690a0 msgr2=0x562806969520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.184+0000 7fe902800700 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628069690a0 0x562806969520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.185+0000 7fe902800700 1 -- 172.21.15.86:0/1773316730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56280673b430 con 0x562806967e60 2024-03-22T19:13:09.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.185+0000 7fe901e00700 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628069690a0 0x562806969520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.185+0000 7fe902800700 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x5628069682a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0013330 tx=0x7fe8f0013640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae785dee071928d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.185+0000 7fe902800700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe8e40a0b60 0x7fe8e40a3020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000cfd0 tx=0x7fe8f0018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.185+0000 7fe8eb000700 1 -- 172.21.15.86:0/1773316730 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0016a70 con 0x562806967e60 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe8eb000700 1 -- 172.21.15.86:0/1773316730 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8f0016bd0 con 0x562806967e60 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe8eb000700 1 -- 172.21.15.86:0/1773316730 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0020e80 con 0x562806967e60 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 -- 172.21.15.86:0/1773316730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 msgr2=0x5628069682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x5628069682a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0013330 tx=0x7fe8f0013640 comp rx=0 tx=0).stop 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 -- 172.21.15.86:0/1773316730 shutdown_connections 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806969a60 0x5628069702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x5628069682a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/1773316730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628069690a0 0x562806969520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.186+0000 7fe9182f0b80 1 -- 172.21.15.86:0/1773316730 >> 172.21.15.86:0/1773316730 conn(0x5628068dd2a0 msgr2=0x562806962e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.187+0000 7fe9182f0b80 1 -- 172.21.15.86:0/1773316730 shutdown_connections 2024-03-22T19:13:09.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.187+0000 7fe9182f0b80 1 -- 172.21.15.86:0/1773316730 wait complete. 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.187+0000 7fe9182f0b80 1 -- start start 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.188+0000 7fe9182f0b80 1 Processor -- start 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806971d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806855e00 con 0x562806969a60 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806855aa0 con 0x562806967e60 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56280673d720 con 0x5628069690a0 2024-03-22T19:13:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.189+0000 7fe9182f0b80 10 client.0 _ll_register_callbacks cb 0x5628066c1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806971d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806971d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48432/0 (socket says 172.21.15.86:48432) 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 -- 172.21.15.86:0/3895179299 learned_addr learned my addr 172.21.15.86:0/3895179299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe902800700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 -- 172.21.15.86:0/3895179299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 msgr2=0x5628069722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901400700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 -- 172.21.15.86:0/3895179299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 msgr2=0x5628069792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901e00700 1 -- 172.21.15.86:0/3895179299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc0714f0 con 0x562806967e60 2024-03-22T19:13:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.190+0000 7fe901400700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.195+0000 7fe901e00700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806971d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0642a0 tx=0x7fe8fc06faf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ac770ec2c92d6d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.195+0000 7fe8df400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.195+0000 7fe8df400700 1 -- 172.21.15.86:0/3895179299 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc076070 con 0x562806967e60 2024-03-22T19:13:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.195+0000 7fe8df400700 1 -- 172.21.15.86:0/3895179299 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe8fc06e9d0 con 0x562806967e60 2024-03-22T19:13:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.195+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56280695acb0 con 0x562806967e60 2024-03-22T19:13:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.196+0000 7fe8df400700 1 -- 172.21.15.86:0/3895179299 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8fc06eb90 con 0x562806967e60 2024-03-22T19:13:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.196+0000 7fe9182f0b80 10 client.25940 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.196+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe8f000cc10 con 0x562806967e60 2024-03-22T19:13:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.196+0000 7fe8df400700 1 -- 172.21.15.86:0/3895179299 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8fc08a020 con 0x562806967e60 2024-03-22T19:13:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.196+0000 7fe8df400700 1 -- 172.21.15.86:0/3895179299 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8fc07b400 con 0x562806967e60 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe9182f0b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe9182f0b80 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe9182f0b80 20 client.25940 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x56280697db40 0x56280697ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe9182f0b80 4 client.25940 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628068b2780 con 0x56280697db40 2024-03-22T19:13:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.197+0000 7fe901400700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x56280697db40 0x56280697ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.200+0000 7fe8df400700 10 client.25940 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.200+0000 7fe901400700 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x56280697db40 0x56280697ff60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf61f56913269c3 server_cookie=bdfddd5bd3aae0ab in_seq=0 out_seq=0 2024-03-22T19:13:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.200+0000 7fe8df400700 1 -- 172.21.15.86:0/3895179299 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5628068b2780 con 0x56280697db40 2024-03-22T19:13:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.201+0000 7fe8df400700 10 client.25940 handle_command_reply: tid=0 2024-03-22T19:13:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.201+0000 7fe9182f0b80 1 client.25940 shutdown 2024-03-22T19:13:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 msgr2=0x562806971d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806971d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0642a0 tx=0x7fe8fc06faf0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 shutdown_connections 2024-03-22T19:13:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x56280697db40 0x56280697ff60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806971d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3895179299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe8df400700 0 client.25940 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 >> 172.21.15.86:0/3895179299 conn(0x5628068dd2a0 msgr2=0x5628069623d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.207+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 shutdown_connections 2024-03-22T19:13:09.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.208+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3895179299 wait complete. 2024-03-22T19:13:09.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.208+0000 7fe9182f0b80 20 client.25940 trim_cache size 0 max 16384 2024-03-22T19:13:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.242+0000 7fe9182f0b80 1 Processor -- start 2024-03-22T19:13:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.243+0000 7fe9182f0b80 1 -- start start 2024-03-22T19:13:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.247+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806967e60 0x562806977260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.247+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628069690a0 0x562806971b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.247+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x562806972070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.247+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56280695f7c0 con 0x562806969a60 2024-03-22T19:13:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.247+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806971910 con 0x5628069690a0 2024-03-22T19:13:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.247+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806962040 con 0x562806967e60 2024-03-22T19:13:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.249+0000 7fe902800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x562806972070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.249+0000 7fe902800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x562806972070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36974/0 (socket says 172.21.15.86:36974) 2024-03-22T19:13:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.249+0000 7fe902800700 1 -- 172.21.15.86:0/4067846936 learned_addr learned my addr 172.21.15.86:0/4067846936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.251+0000 7fe901e00700 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806967e60 0x562806977260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.250+0000 7fe902800700 1 -- 172.21.15.86:0/4067846936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806967e60 msgr2=0x562806977260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.251+0000 7fe902800700 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806967e60 0x562806977260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.251+0000 7fe902800700 1 -- 172.21.15.86:0/4067846936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628069690a0 msgr2=0x562806971b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.251+0000 7fe902800700 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628069690a0 0x562806971b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.251+0000 7fe902800700 1 -- 172.21.15.86:0/4067846936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628066e7be0 con 0x562806969a60 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.252+0000 7fe901e00700 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806967e60 0x562806977260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.252+0000 7fe902800700 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x562806972070 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000cf00 tx=0x7fe8f000c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81af6925bf84e5e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.253+0000 7fe8eba00700 1 -- 172.21.15.86:0/4067846936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0016a70 con 0x562806969a60 2024-03-22T19:13:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.253+0000 7fe8eba00700 1 -- 172.21.15.86:0/4067846936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8f0020a30 con 0x562806969a60 2024-03-22T19:13:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.253+0000 7fe8eba00700 1 -- 172.21.15.86:0/4067846936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f002a6c0 con 0x562806969a60 2024-03-22T19:13:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 -- 172.21.15.86:0/4067846936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 msgr2=0x562806972070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x562806972070 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000cf00 tx=0x7fe8f000c5c0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 -- 172.21.15.86:0/4067846936 shutdown_connections 2024-03-22T19:13:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x562806972070 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562806967e60 0x562806977260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/4067846936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628069690a0 0x562806971b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 -- 172.21.15.86:0/4067846936 >> 172.21.15.86:0/4067846936 conn(0x5628068dd2a0 msgr2=0x562806961930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.255+0000 7fe9182f0b80 1 -- 172.21.15.86:0/4067846936 shutdown_connections 2024-03-22T19:13:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.256+0000 7fe9182f0b80 1 -- 172.21.15.86:0/4067846936 wait complete. 2024-03-22T19:13:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.256+0000 7fe9182f0b80 1 -- start start 2024-03-22T19:13:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.257+0000 7fe9182f0b80 1 Processor -- start 2024-03-22T19:13:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.257+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806974190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.257+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.257+0000 7fe9182f0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.258+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806974e00 con 0x562806969a60 2024-03-22T19:13:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.258+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628069e0bd0 con 0x562806967e60 2024-03-22T19:13:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.258+0000 7fe9182f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628069e0d90 con 0x5628069690a0 2024-03-22T19:13:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.258+0000 7fe9182f0b80 10 client.0 _ll_register_callbacks cb 0x7fe8fc072c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.258+0000 7fe902800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.259+0000 7fe902800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069e0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36990/0 (socket says 172.21.15.86:36990) 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.259+0000 7fe902800700 1 -- 172.21.15.86:0/3493254864 learned_addr learned my addr 172.21.15.86:0/3493254864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.259+0000 7fe901e00700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806974190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.259+0000 7fe901400700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.260+0000 7fe902800700 1 -- 172.21.15.86:0/3493254864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 msgr2=0x5628069746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.260+0000 7fe902800700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.260+0000 7fe902800700 1 -- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 msgr2=0x562806974190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.260+0000 7fe902800700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806974190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.260+0000 7fe902800700 1 -- 172.21.15.86:0/3493254864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56280673b430 con 0x562806969a60 2024-03-22T19:13:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.260+0000 7fe901400700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.261+0000 7fe902800700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069e0660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0020f20 tx=0x7fe8f000c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff262e984af1a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.261+0000 7fe8df400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.262+0000 7fe8df400700 1 -- 172.21.15.86:0/3493254864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0013eb0 con 0x562806969a60 2024-03-22T19:13:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.262+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8f0016bd0 con 0x562806969a60 2024-03-22T19:13:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.262+0000 7fe8df400700 1 -- 172.21.15.86:0/3493254864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe8f002a480 con 0x562806969a60 2024-03-22T19:13:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.263+0000 7fe8df400700 1 -- 172.21.15.86:0/3493254864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f0013eb0 con 0x562806969a60 2024-03-22T19:13:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.264+0000 7fe9182f0b80 10 client.15891 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:09.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.264+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56280687ff20 con 0x562806969a60 2024-03-22T19:13:09.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.269+0000 7fe8df400700 1 -- 172.21.15.86:0/3493254864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8f0016bd0 con 0x562806969a60 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe8df400700 1 -- 172.21.15.86:0/3493254864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8f001e030 con 0x562806969a60 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe9182f0b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe9182f0b80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe9182f0b80 20 client.15891 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628069e2540 0x5628069e4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe9182f0b80 4 client.15891 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:13:09.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.272+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562806708d70 con 0x5628069e2540 2024-03-22T19:13:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.273+0000 7fe901e00700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628069e2540 0x5628069e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.274+0000 7fe8df400700 10 client.15891 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.275+0000 7fe901e00700 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628069e2540 0x5628069e4960 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173035145f192edd server_cookie=2c1c2de9aba4728a in_seq=0 out_seq=0 2024-03-22T19:13:09.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.276+0000 7fe8df400700 1 -- 172.21.15.86:0/3493254864 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fe8fc08a080 con 0x5628069e2540 2024-03-22T19:13:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.276+0000 7fe8df400700 10 client.15891 handle_command_reply: tid=0 2024-03-22T19:13:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.276+0000 7fe9182f0b80 1 client.15891 shutdown 2024-03-22T19:13:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.278+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 msgr2=0x5628069e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.278+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069e0660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0020f20 tx=0x7fe8f000c530 comp rx=0 tx=0).stop 2024-03-22T19:13:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 shutdown_connections 2024-03-22T19:13:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628069e2540 0x5628069e4960 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562806969a60 0x5628069e0660 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562806967e60 0x562806974190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe9182f0b80 1 --2- 172.21.15.86:0/3493254864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628069690a0 0x5628069746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 >> 172.21.15.86:0/3493254864 conn(0x5628068dd2a0 msgr2=0x562806971b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.281+0000 7fe8df400700 0 client.15891 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.282+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 shutdown_connections 2024-03-22T19:13:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.282+0000 7fe9182f0b80 1 -- 172.21.15.86:0/3493254864 wait complete. 2024-03-22T19:13:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.283+0000 7fe9182f0b80 20 client.15891 trim_cache size 0 max 16384 2024-03-22T19:13:09.285 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:13:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:13:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "490a0920-5501-4daa-8876-1f72de17813f", 2024-03-22T19:13:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:13:09.286 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:13:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.285+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe8e40a0b60 msgr2=0x7fe8e40a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.285+0000 7fe903200700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe8e40a0b60 0x7fe8e40a3020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000cfd0 tx=0x7fe8f0018000 comp rx=0 tx=0).stop 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.285+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 msgr2=0x7fe8f8143980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.285+0000 7fe903200700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f8143980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b5c0 tx=0x7fe8ec003c00 comp rx=0 tx=0).stop 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.286+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 shutdown_connections 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.286+0000 7fe903200700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe8e40a0b60 0x7fe8e40a3020 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.286+0000 7fe903200700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f80a6780 0x7fe8f814a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.286+0000 7fe903200700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8f80a4c90 0x7fe8f8143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.286+0000 7fe903200700 1 --2- 172.21.15.86:0/1947307050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f80a5dc0 0x7fe8f8143980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.286+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 >> 172.21.15.86:0/1947307050 conn(0x7fe8f80a0120 msgr2=0x7fe8f80a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.287+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 shutdown_connections 2024-03-22T19:13:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.287+0000 7fe903200700 1 -- 172.21.15.86:0/1947307050 wait complete. 2024-03-22T19:13:09.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:13:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:09 smithi167 ceph-mon[47728]: pgmap v351: 97 pgs: 97 active+clean; 5.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 260 MiB/s wr, 95 op/s 2024-03-22T19:13:09.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 -- 172.21.15.86:0/1265346459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c1040d0 msgr2=0x7f934c104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 --2- 172.21.15.86:0/1265346459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c1040d0 0x7f934c104550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9338009990 tx=0x7f9338009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 -- 172.21.15.86:0/1265346459 shutdown_connections 2024-03-22T19:13:09.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 --2- 172.21.15.86:0/1265346459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f934c104a90 0x7f934c10b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 --2- 172.21.15.86:0/1265346459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c102f00 0x7f934c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 --2- 172.21.15.86:0/1265346459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c1040d0 0x7f934c104550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.687+0000 7f9351e00700 1 -- 172.21.15.86:0/1265346459 >> 172.21.15.86:0/1265346459 conn(0x7f934c0fe4c0 msgr2=0x7f934c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.688+0000 7f9351e00700 1 -- 172.21.15.86:0/1265346459 shutdown_connections 2024-03-22T19:13:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.688+0000 7f9351e00700 1 -- 172.21.15.86:0/1265346459 wait complete. 2024-03-22T19:13:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.689+0000 7f9351e00700 1 Processor -- start 2024-03-22T19:13:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.691+0000 7f9351e00700 1 -- start start 2024-03-22T19:13:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.692+0000 7f9351e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c102f00 0x7f934c194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.692+0000 7f9351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c1040d0 0x7f934c194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.693+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c102f00 0x7f934c194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.693+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c102f00 0x7f934c194a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:50136/0 (socket says 172.21.15.86:50136) 2024-03-22T19:13:09.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.693+0000 7f934aa00700 1 -- 172.21.15.86:0/3380619094 learned_addr learned my addr 172.21.15.86:0/3380619094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.693+0000 7f934a000700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c1040d0 0x7f934c194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.693+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f934c104a90 0x7f934c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.694+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c19c540 con 0x7f934c104a90 2024-03-22T19:13:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.694+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c19c6b0 con 0x7f934c1040d0 2024-03-22T19:13:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.694+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1ad830 con 0x7f934c102f00 2024-03-22T19:13:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.694+0000 7f934b400700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f934c104a90 0x7f934c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.695+0000 7f934a000700 1 -- 172.21.15.86:0/3380619094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c102f00 msgr2=0x7f934c194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.695+0000 7f934a000700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c102f00 0x7f934c194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.695+0000 7f934a000700 1 -- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f934c104a90 msgr2=0x7f934c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.695+0000 7f934a000700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f934c104a90 0x7f934c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.695+0000 7f934a000700 1 -- 172.21.15.86:0/3380619094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380095f0 con 0x7f934c1040d0 2024-03-22T19:13:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.696+0000 7f934a000700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c1040d0 0x7f934c194f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9338000c00 tx=0x7f9338009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.696+0000 7f9350a00700 1 -- 172.21.15.86:0/3380619094 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933801d070 con 0x7f934c1040d0 2024-03-22T19:13:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.696+0000 7f9350a00700 1 -- 172.21.15.86:0/3380619094 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9338021410 con 0x7f934c1040d0 2024-03-22T19:13:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.696+0000 7f9350a00700 1 -- 172.21.15.86:0/3380619094 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338017650 con 0x7f934c1040d0 2024-03-22T19:13:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.697+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c1adab0 con 0x7f934c1040d0 2024-03-22T19:13:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.697+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c1ade80 con 0x7f934c1040d0 2024-03-22T19:13:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.704+0000 7f93658e7b80 1 Processor -- start 2024-03-22T19:13:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.706+0000 7f93658e7b80 1 -- start start 2024-03-22T19:13:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.707+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 0x55e05206a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.707+0000 7f9350a00700 1 -- 172.21.15.86:0/3380619094 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f933800fa50 con 0x7f934c1040d0 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e05206a580 0x55e05206ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05206b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f93658e7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051fc2990 con 0x55e05206b420 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f93658e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051fc0350 con 0x55e052069c20 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f93658e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051fbbc40 con 0x55e05206a580 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f934b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 0x55e05206a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f934b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 0x55e05206a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48494/0 (socket says 172.21.15.86:48494) 2024-03-22T19:13:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.709+0000 7f934b400700 1 -- 172.21.15.86:0/2760679421 learned_addr learned my addr 172.21.15.86:0/2760679421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.710+0000 7f934aa00700 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e05206a580 0x55e05206ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.710+0000 7f934b400700 1 -- 172.21.15.86:0/2760679421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e05206a580 msgr2=0x55e05206ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.710+0000 7f934b400700 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e05206a580 0x55e05206ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.710+0000 7f934b400700 1 -- 172.21.15.86:0/2760679421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 msgr2=0x55e05206b8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.710+0000 7f934b400700 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05206b8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.710+0000 7f934b400700 1 -- 172.21.15.86:0/2760679421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e051e3d3b0 con 0x55e052069c20 2024-03-22T19:13:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.711+0000 7f934b400700 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 0x55e05206a040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f93440098f0 tx=0x7f9344009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb1aaeda9d655a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.711+0000 7f9337e00700 1 -- 172.21.15.86:0/2760679421 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344019070 con 0x55e052069c20 2024-03-22T19:13:09.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.711+0000 7f9337e00700 1 -- 172.21.15.86:0/2760679421 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f934400f040 con 0x55e052069c20 2024-03-22T19:13:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.711+0000 7f9337e00700 1 -- 172.21.15.86:0/2760679421 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344007c00 con 0x55e052069c20 2024-03-22T19:13:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.711+0000 7f9350a00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f93300a0c90 0x7f93300a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.712+0000 7f9350a00700 1 -- 172.21.15.86:0/3380619094 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f93380cd180 con 0x7f934c1040d0 2024-03-22T19:13:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.712+0000 7f934aa00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f93300a0c90 0x7f93300a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.712+0000 7f93658e7b80 1 -- 172.21.15.86:0/2760679421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 msgr2=0x55e05206a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.712+0000 7f93658e7b80 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 0x55e05206a040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f93440098f0 tx=0x7f9344009c00 comp rx=0 tx=0).stop 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f934aa00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f93300a0c90 0x7f93300a3150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f933c005950 tx=0x7f933c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 -- 172.21.15.86:0/2760679421 shutdown_connections 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05206b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e052069c20 0x55e05206a040 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 --2- 172.21.15.86:0/2760679421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e05206a580 0x55e05206ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 -- 172.21.15.86:0/2760679421 >> 172.21.15.86:0/2760679421 conn(0x55e051fdf220 msgr2=0x55e052064bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 -- 172.21.15.86:0/2760679421 shutdown_connections 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.713+0000 7f93658e7b80 1 -- 172.21.15.86:0/2760679421 wait complete. 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.714+0000 7f93658e7b80 1 -- start start 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.714+0000 7f93658e7b80 1 Processor -- start 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.715+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052073c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.716+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e052074190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.716+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.716+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052073c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:50154/0 (socket says 172.21.15.86:50154) 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.716+0000 7f934aa00700 1 -- 172.21.15.86:0/947603774 learned_addr learned my addr 172.21.15.86:0/947603774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.716+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05207b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.717+0000 7f934a000700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e052074190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.717+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051f57d80 con 0x55e05206b420 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.717+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051f57a20 con 0x55e05206a580 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.717+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051e3f6a0 con 0x55e052069c20 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.717+0000 7f93658e7b80 10 client.0 _ll_register_callbacks cb 0x55e051dc33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.717+0000 7f934b400700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05207b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934aa00700 1 -- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 msgr2=0x55e052074190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934aa00700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e052074190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934aa00700 1 -- 172.21.15.86:0/947603774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 msgr2=0x55e05207b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934aa00700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05207b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934aa00700 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9344003c50 con 0x55e052069c20 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934a000700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e052074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.718+0000 7f934b400700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05207b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.719+0000 7f934aa00700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052073c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f933c01d6c0 tx=0x7f933c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b67eaeb1940c857 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.719+0000 7f9334c00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:09 smithi131 ceph-mon[78559]: pgmap v351: 97 pgs: 97 active+clean; 5.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 260 MiB/s wr, 95 op/s 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.719+0000 7f9334c00700 1 -- 172.21.15.86:0/947603774 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c022070 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.719+0000 7f9334c00700 1 -- 172.21.15.86:0/947603774 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f933c01de10 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.720+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e05205cad0 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.720+0000 7f9334c00700 1 -- 172.21.15.86:0/947603774 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933c017070 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.720+0000 7f93658e7b80 10 client.25939 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.720+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e051de9b60 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.721+0000 7f9334c00700 1 -- 172.21.15.86:0/947603774 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e05205cad0 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.721+0000 7f9334c00700 1 -- 172.21.15.86:0/947603774 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f933c030070 con 0x55e052069c20 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.721+0000 7f93658e7b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.721+0000 7f93658e7b80 10 client.25939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:09.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.721+0000 7f93658e7b80 20 client.25939 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.721+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e05207f9f0 0x55e052081e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.722+0000 7f93658e7b80 4 client.25939 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.722+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e051fb4700 con 0x55e05207f9f0 2024-03-22T19:13:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.722+0000 7f934a000700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e05207f9f0 0x55e052081e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.723+0000 7f934a000700 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e05207f9f0 0x55e052081e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68b6353f82270e6 server_cookie=1190123cccfad04d in_seq=0 out_seq=0 2024-03-22T19:13:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.723+0000 7f9334c00700 10 client.25939 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.724+0000 7f9334c00700 1 -- 172.21.15.86:0/947603774 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e051fb4700 con 0x55e05207f9f0 2024-03-22T19:13:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.725+0000 7f9334c00700 10 client.25939 handle_command_reply: tid=0 2024-03-22T19:13:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.727+0000 7f93658e7b80 1 client.25939 shutdown 2024-03-22T19:13:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.727+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 msgr2=0x55e052073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.727+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052073c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f933c01d6c0 tx=0x7f933c0047c0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.727+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 shutdown_connections 2024-03-22T19:13:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e05207f9f0 0x55e052081e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e05207b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052073c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 --2- 172.21.15.86:0/947603774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e052074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f9334c00700 0 client.25939 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 >> 172.21.15.86:0/947603774 conn(0x55e051fdf220 msgr2=0x55e0520641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 shutdown_connections 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.728+0000 7f93658e7b80 1 -- 172.21.15.86:0/947603774 wait complete. 2024-03-22T19:13:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.729+0000 7f93658e7b80 20 client.25939 trim_cache size 0 max 16384 2024-03-22T19:13:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:09 smithi086 ceph-mon[38032]: pgmap v351: 97 pgs: 97 active+clean; 5.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 260 MiB/s wr, 95 op/s 2024-03-22T19:13:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.767+0000 7f93658e7b80 1 Processor -- start 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 -- start start 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052074110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e05207b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e0520641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05205f210 con 0x55e05206b420 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05206a960 con 0x55e05206a580 2024-03-22T19:13:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.768+0000 7f93658e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05207c350 con 0x55e052069c20 2024-03-22T19:13:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:50170/0 (socket says 172.21.15.86:50170) 2024-03-22T19:13:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934aa00700 1 -- 172.21.15.86:0/308443648 learned_addr learned my addr 172.21.15.86:0/308443648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934b400700 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e0520641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934aa00700 1 -- 172.21.15.86:0/308443648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 msgr2=0x55e05207b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934aa00700 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e05207b190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.770+0000 7f934aa00700 1 -- 172.21.15.86:0/308443648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 msgr2=0x55e0520641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.771+0000 7f934aa00700 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e0520641f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.771+0000 7f934aa00700 1 -- 172.21.15.86:0/308443648 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e051f81ea0 con 0x55e052069c20 2024-03-22T19:13:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.771+0000 7f934aa00700 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052074110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f933c009ad0 tx=0x7f933c00f800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51c7f0f4f441de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.772+0000 7f9341200700 1 -- 172.21.15.86:0/308443648 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c022030 con 0x55e052069c20 2024-03-22T19:13:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.772+0000 7f9341200700 1 -- 172.21.15.86:0/308443648 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933c00f8d0 con 0x55e052069c20 2024-03-22T19:13:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.772+0000 7f9341200700 1 -- 172.21.15.86:0/308443648 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c026a00 con 0x55e052069c20 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 -- 172.21.15.86:0/308443648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 msgr2=0x55e052074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052074110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f933c009ad0 tx=0x7f933c00f800 comp rx=0 tx=0).stop 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 -- 172.21.15.86:0/308443648 shutdown_connections 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e05206b420 0x55e0520641f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e052074110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 --2- 172.21.15.86:0/308443648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206a580 0x55e05207b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.773+0000 7f93658e7b80 1 -- 172.21.15.86:0/308443648 >> 172.21.15.86:0/308443648 conn(0x55e051fdf220 msgr2=0x55e052073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.775+0000 7f93658e7b80 1 -- 172.21.15.86:0/308443648 shutdown_connections 2024-03-22T19:13:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.775+0000 7f93658e7b80 1 -- 172.21.15.86:0/308443648 wait complete. 2024-03-22T19:13:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.776+0000 7f93658e7b80 1 -- start start 2024-03-22T19:13:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.777+0000 7f93658e7b80 1 Processor -- start 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.777+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e05207b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.777+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206b420 0x55e052079700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.777+0000 7f93658e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e052079c40 0x55e05207a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.777+0000 7f93658e7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0520d1540 con 0x55e052079c40 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.777+0000 7f93658e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0520d16b0 con 0x55e05206b420 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.778+0000 7f93658e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0520e2880 con 0x55e052069c20 2024-03-22T19:13:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.778+0000 7f93658e7b80 10 client.0 _ll_register_callbacks cb 0x7f933c01d550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.778+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e05207b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.779+0000 7f934aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e05207b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:50182/0 (socket says 172.21.15.86:50182) 2024-03-22T19:13:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.779+0000 7f934aa00700 1 -- 172.21.15.86:0/1009313449 learned_addr learned my addr 172.21.15.86:0/1009313449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.779+0000 7f934a000700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206b420 0x55e052079700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.779+0000 7f934b400700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e052079c40 0x55e05207a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.780+0000 7f934b400700 1 -- 172.21.15.86:0/1009313449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 msgr2=0x55e05207b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.780+0000 7f934b400700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e05207b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.780+0000 7f934b400700 1 -- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206b420 msgr2=0x55e052079700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.781+0000 7f934b400700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206b420 0x55e052079700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.781+0000 7f934b400700 1 -- 172.21.15.86:0/1009313449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e051de9b60 con 0x55e052079c40 2024-03-22T19:13:09.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.781+0000 7f934aa00700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e05207b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:09.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.781+0000 7f934a000700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206b420 0x55e052079700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:09.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.782+0000 7f934b400700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e052079c40 0x55e05207a0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55e052065de0 tx=0x7f9344007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e198fc57ff5db2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:09.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.782+0000 7f9334c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:09.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.782+0000 7f9334c00700 1 -- 172.21.15.86:0/1009313449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344019040 con 0x55e052079c40 2024-03-22T19:13:09.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.783+0000 7f9334c00700 1 -- 172.21.15.86:0/1009313449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9344004880 con 0x55e052079c40 2024-03-22T19:13:09.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.783+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f934400f040 con 0x55e052079c40 2024-03-22T19:13:09.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.784+0000 7f9334c00700 1 -- 172.21.15.86:0/1009313449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9344004a40 con 0x55e052079c40 2024-03-22T19:13:09.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.784+0000 7f93658e7b80 10 client.15903 fetch_fsmap learned FSMap version 29 2024-03-22T19:13:09.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.784+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e051e41cd0 con 0x55e052079c40 2024-03-22T19:13:09.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f9334c00700 1 -- 172.21.15.86:0/1009313449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f934400f040 con 0x55e052079c40 2024-03-22T19:13:09.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f9334c00700 1 -- 172.21.15.86:0/1009313449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f934401e440 con 0x55e052079c40 2024-03-22T19:13:09.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f93658e7b80 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-03-22T19:13:09.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f93658e7b80 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:09.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f93658e7b80 20 client.15903 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:09.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f93658e7b80 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e0520e42e0 0x55e0520e6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:09.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f93658e7b80 4 client.15903 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:13:09.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.786+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e051e0acf0 con 0x55e0520e42e0 2024-03-22T19:13:09.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.787+0000 7f934aa00700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e0520e42e0 0x55e0520e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:09.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f9334c00700 10 client.15903 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f934aa00700 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e0520e42e0 0x55e0520e6700 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5128829530c6604 server_cookie=169c8980a39cd97d in_seq=0 out_seq=0 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f9334c00700 1 -- 172.21.15.86:0/1009313449 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e051e0acf0 con 0x55e0520e42e0 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f9334c00700 10 client.15903 handle_command_reply: tid=0 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 client.15903 shutdown 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e052079c40 msgr2=0x55e05207a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e052079c40 0x55e05207a0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55e052065de0 tx=0x7f9344007f70 comp rx=0 tx=0).stop 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 shutdown_connections 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55e0520e42e0 0x55e0520e6700 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e052079c40 0x55e05207a0c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e052069c20 0x55e05207b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.788+0000 7f93658e7b80 1 --2- 172.21.15.86:0/1009313449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e05206b420 0x55e052079700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.789+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 >> 172.21.15.86:0/1009313449 conn(0x55e051fdf220 msgr2=0x55e0520641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.789+0000 7f9334c00700 0 client.15903 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:09.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.789+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 shutdown_connections 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.789+0000 7f93658e7b80 1 -- 172.21.15.86:0/1009313449 wait complete. 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.789+0000 7f93658e7b80 20 client.15903 trim_cache size 0 max 16384 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stdout: "490a0920-5501-4daa-8876-1f72de17813f": { 2024-03-22T19:13:09.792 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "490a0920-5501-4daa-8876-1f72de17813f", 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f93300a0c90 msgr2=0x7f93300a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f93300a0c90 0x7f93300a3150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f933c005950 tx=0x7f933c009260 comp rx=0 tx=0).stop 2024-03-22T19:13:09.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c1040d0 msgr2=0x7f934c194f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c1040d0 0x7f934c194f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9338000c00 tx=0x7f9338009580 comp rx=0 tx=0).stop 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 shutdown_connections 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f93300a0c90 0x7f93300a3150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f934c104a90 0x7f934c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934c102f00 0x7f934c194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 --2- 172.21.15.86:0/3380619094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c1040d0 0x7f934c194f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.790+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 >> 172.21.15.86:0/3380619094 conn(0x7f934c0fe4c0 msgr2=0x7f934c0772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.796+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 shutdown_connections 2024-03-22T19:13:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:09.796+0000 7f9351e00700 1 -- 172.21.15.86:0/3380619094 wait complete. 2024-03-22T19:13:09.820 INFO:tasks.cephfs.filesystem:scrub status for tag:490a0920-5501-4daa-8876-1f72de17813f - {'path': '/', 'tag': '490a0920-5501-4daa-8876-1f72de17813f', 'options': 'recursive,force'} 2024-03-22T19:13:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:13:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:10.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:13:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:10.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:10.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:10.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:11 smithi167 ceph-mon[47728]: pgmap v352: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 248 MiB/s wr, 99 op/s 2024-03-22T19:13:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:11 smithi131 ceph-mon[78559]: pgmap v352: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 248 MiB/s wr, 99 op/s 2024-03-22T19:13:11.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:11 smithi086 ceph-mon[38032]: pgmap v352: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 248 MiB/s wr, 99 op/s 2024-03-22T19:13:13.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:13 smithi167 ceph-mon[47728]: pgmap v353: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 292 MiB/s wr, 103 op/s 2024-03-22T19:13:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:13 smithi131 ceph-mon[78559]: pgmap v353: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 292 MiB/s wr, 103 op/s 2024-03-22T19:13:13.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:13 smithi086 ceph-mon[38032]: pgmap v353: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 292 MiB/s wr, 103 op/s 2024-03-22T19:13:15.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:15 smithi167 ceph-mon[47728]: pgmap v354: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 229 MiB/s wr, 86 op/s 2024-03-22T19:13:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:15 smithi131 ceph-mon[78559]: pgmap v354: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 229 MiB/s wr, 86 op/s 2024-03-22T19:13:15.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:15 smithi086 ceph-mon[38032]: pgmap v354: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 229 MiB/s wr, 86 op/s 2024-03-22T19:13:17.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:17 smithi131 ceph-mon[78559]: pgmap v355: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 267 MiB/s wr, 106 op/s 2024-03-22T19:13:17.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:17 smithi086 ceph-mon[38032]: pgmap v355: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 267 MiB/s wr, 106 op/s 2024-03-22T19:13:17.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:17 smithi167 ceph-mon[47728]: pgmap v355: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 267 MiB/s wr, 106 op/s 2024-03-22T19:13:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:13:18.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:13:18.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:13:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:13:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:13:18.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:13:18.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:13:18.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:13:18.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:13:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:19 smithi086 ceph-mon[38032]: pgmap v356: 97 pgs: 97 active+clean; 8.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 266 MiB/s wr, 97 op/s 2024-03-22T19:13:19.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:19 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:13:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:19 smithi167 ceph-mon[47728]: pgmap v356: 97 pgs: 97 active+clean; 8.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 266 MiB/s wr, 97 op/s 2024-03-22T19:13:19.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:19 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:13:19.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:19 smithi131 ceph-mon[78559]: pgmap v356: 97 pgs: 97 active+clean; 8.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 266 MiB/s wr, 97 op/s 2024-03-22T19:13:19.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:19 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:13:21.109 INFO:tasks.workunit.client.0.smithi086.stdout:fs setup took 44 secs 2024-03-22T19:13:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:21 smithi167 ceph-mon[47728]: pgmap v357: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 255 MiB/s wr, 102 op/s 2024-03-22T19:13:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:13:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:13:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:13:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:21 smithi131 ceph-mon[78559]: pgmap v357: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 255 MiB/s wr, 102 op/s 2024-03-22T19:13:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:13:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:13:21.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:13:21.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:21 smithi086 ceph-mon[38032]: pgmap v357: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 255 MiB/s wr, 102 op/s 2024-03-22T19:13:21.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:13:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:13:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:13:23.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:23 smithi167 ceph-mon[47728]: pgmap v358: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 313 MiB/s wr, 109 op/s 2024-03-22T19:13:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:23 smithi131 ceph-mon[78559]: pgmap v358: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 313 MiB/s wr, 109 op/s 2024-03-22T19:13:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:23 smithi086 ceph-mon[38032]: pgmap v358: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 313 MiB/s wr, 109 op/s 2024-03-22T19:13:25.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:25 smithi167 ceph-mon[47728]: pgmap v359: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 259 MiB/s wr, 95 op/s 2024-03-22T19:13:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:25 smithi131 ceph-mon[78559]: pgmap v359: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 259 MiB/s wr, 95 op/s 2024-03-22T19:13:25.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:25 smithi086 ceph-mon[38032]: pgmap v359: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 259 MiB/s wr, 95 op/s 2024-03-22T19:13:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:27 smithi167 ceph-mon[47728]: pgmap v360: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 313 MiB/s wr, 110 op/s 2024-03-22T19:13:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:27 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:27 smithi131 ceph-mon[78559]: pgmap v360: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 313 MiB/s wr, 110 op/s 2024-03-22T19:13:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:27 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:27.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:27 smithi086 ceph-mon[38032]: pgmap v360: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 313 MiB/s wr, 110 op/s 2024-03-22T19:13:27.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:27 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:28.544 INFO:tasks.workunit.client.0.smithi086.stdout:Syncing()...7 sec 2024-03-22T19:13:28.546 INFO:tasks.workunit.client.0.smithi086.stdout:Starting Actual Benchmark At: Fri Mar 22 19:13:28 2024 2024-03-22T19:13:28.546 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:13:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:29 smithi167 ceph-mon[47728]: pgmap v361: 97 pgs: 97 active+clean; 12 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 316 MiB/s wr, 102 op/s 2024-03-22T19:13:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:29 smithi131 ceph-mon[78559]: pgmap v361: 97 pgs: 97 active+clean; 12 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 316 MiB/s wr, 102 op/s 2024-03-22T19:13:29.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:29 smithi086 ceph-mon[38032]: pgmap v361: 97 pgs: 97 active+clean; 12 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 316 MiB/s wr, 102 op/s 2024-03-22T19:13:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:31 smithi167 ceph-mon[47728]: pgmap v362: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 300 MiB/s wr, 104 op/s 2024-03-22T19:13:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:31 smithi131 ceph-mon[78559]: pgmap v362: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 300 MiB/s wr, 104 op/s 2024-03-22T19:13:31.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:31 smithi086 ceph-mon[38032]: pgmap v362: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 300 MiB/s wr, 104 op/s 2024-03-22T19:13:32.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:32 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:32.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:32 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:32.763 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:32 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:33.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:33 smithi131 ceph-mon[78559]: pgmap v363: 97 pgs: 97 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 325 MiB/s wr, 2.14k op/s 2024-03-22T19:13:33.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:33 smithi086 ceph-mon[38032]: pgmap v363: 97 pgs: 97 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 325 MiB/s wr, 2.14k op/s 2024-03-22T19:13:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:33 smithi167 ceph-mon[47728]: pgmap v363: 97 pgs: 97 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 325 MiB/s wr, 2.14k op/s 2024-03-22T19:13:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:35 smithi131 ceph-mon[78559]: pgmap v364: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 251 MiB/s wr, 3.35k op/s 2024-03-22T19:13:35.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:35 smithi086 ceph-mon[38032]: pgmap v364: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 251 MiB/s wr, 3.35k op/s 2024-03-22T19:13:35.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:35 smithi167 ceph-mon[47728]: pgmap v364: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 251 MiB/s wr, 3.35k op/s 2024-03-22T19:13:37.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:37 smithi131 ceph-mon[78559]: pgmap v365: 97 pgs: 97 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 251 MiB/s wr, 6.75k op/s 2024-03-22T19:13:37.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:37 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:37.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:37 smithi086 ceph-mon[38032]: pgmap v365: 97 pgs: 97 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 251 MiB/s wr, 6.75k op/s 2024-03-22T19:13:37.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:37 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:37.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:37 smithi167 ceph-mon[47728]: pgmap v365: 97 pgs: 97 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 251 MiB/s wr, 6.75k op/s 2024-03-22T19:13:37.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:37 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:13:39.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:39 smithi131 ceph-mon[78559]: pgmap v366: 97 pgs: 97 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 213 MiB/s wr, 11.92k op/s 2024-03-22T19:13:39.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:39 smithi086 ceph-mon[38032]: pgmap v366: 97 pgs: 97 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 213 MiB/s wr, 11.92k op/s 2024-03-22T19:13:39.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:13:39.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:39 smithi167 ceph-mon[47728]: pgmap v366: 97 pgs: 97 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 213 MiB/s wr, 11.92k op/s 2024-03-22T19:13:40.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.269+0000 7f9eb5a00700 1 -- 172.21.15.86:0/4214819849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0100e20 msgr2=0x7f9eb0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.269+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/4214819849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0100e20 0x7f9eb0101280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0099c0 tx=0x7f9e9c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 -- 172.21.15.86:0/4214819849 shutdown_connections 2024-03-22T19:13:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/4214819849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb01017c0 0x7f9eb0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/4214819849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb00ffc30 0x7f9eb0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/4214819849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0100e20 0x7f9eb0101280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 -- 172.21.15.86:0/4214819849 >> 172.21.15.86:0/4214819849 conn(0x7f9eb00fb1a0 msgr2=0x7f9eb00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 -- 172.21.15.86:0/4214819849 shutdown_connections 2024-03-22T19:13:40.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.272+0000 7f9eb5a00700 1 -- 172.21.15.86:0/4214819849 wait complete. 2024-03-22T19:13:40.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.274+0000 7f9eb5a00700 1 Processor -- start 2024-03-22T19:13:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.278+0000 7f9eb5a00700 1 -- start start 2024-03-22T19:13:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.278+0000 7f9eb5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb00ffc30 0x7f9eb0198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.278+0000 7f9eb5a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 0x7f9eb01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eb5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb01017c0 0x7f9eb01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eb5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01a09a0 con 0x7f9eb00ffc30 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eb5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01a0ae0 con 0x7f9eb01017c0 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eb5a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01a0c70 con 0x7f9eb0100e20 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eae000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 0x7f9eb01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eae000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 0x7f9eb01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59670/0 (socket says 172.21.15.86:59670) 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eae000700 1 -- 172.21.15.86:0/3314435963 learned_addr learned my addr 172.21.15.86:0/3314435963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eaea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb00ffc30 0x7f9eb0198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.281+0000 7f9eaf400700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb01017c0 0x7f9eb01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.282+0000 7f9eae000700 1 -- 172.21.15.86:0/3314435963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb01017c0 msgr2=0x7f9eb01a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.282+0000 7f9eae000700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb01017c0 0x7f9eb01a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.282+0000 7f9eae000700 1 -- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb00ffc30 msgr2=0x7f9eb0198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.282+0000 7f9eae000700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb00ffc30 0x7f9eb0198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.282+0000 7f9eae000700 1 -- 172.21.15.86:0/3314435963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0095f0 con 0x7f9eb0100e20 2024-03-22T19:13:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.282+0000 7f9eaea00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb00ffc30 0x7f9eb0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.283+0000 7f9eae000700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 0x7f9eb01993b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c003760 tx=0x7f9e9c003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.283+0000 7f9ea7400700 1 -- 172.21.15.86:0/3314435963 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c01d070 con 0x7f9eb0100e20 2024-03-22T19:13:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.283+0000 7f9ea7400700 1 -- 172.21.15.86:0/3314435963 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e9c00bb60 con 0x7f9eb0100e20 2024-03-22T19:13:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.283+0000 7f9ea7400700 1 -- 172.21.15.86:0/3314435963 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00f6e0 con 0x7f9eb0100e20 2024-03-22T19:13:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.283+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb01a0ef0 con 0x7f9eb0100e20 2024-03-22T19:13:40.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.284+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb01a12b0 con 0x7f9eb0100e20 2024-03-22T19:13:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.289+0000 7f9ea7400700 1 -- 172.21.15.86:0/3314435963 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9e9c022470 con 0x7f9eb0100e20 2024-03-22T19:13:40.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.293+0000 7f9ea7400700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9e940a0b40 0x7f9e940a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.293+0000 7f9ea7400700 1 -- 172.21.15.86:0/3314435963 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9e9c0cd460 con 0x7f9eb0100e20 2024-03-22T19:13:40.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.294+0000 7f9eaea00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9e940a0b40 0x7f9e940a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 Processor -- start 2024-03-22T19:13:40.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 -- start start 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5baed10 0x55bcf5baf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5afe990 con 0x55bcf5baed10 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5afc350 con 0x55bcf5bafc00 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9ec943fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5af7c40 con 0x55bcf5bae3b0 2024-03-22T19:13:40.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.296+0000 7f9eaea00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9e940a0b40 0x7f9e940a3000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0100c80 tx=0x7f9ea0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50340/0 (socket says 172.21.15.86:50340) 2024-03-22T19:13:40.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 -- 172.21.15.86:0/1917089639 learned_addr learned my addr 172.21.15.86:0/1917089639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:40.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 -- 172.21.15.86:0/1917089639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bae3b0 msgr2=0x55bcf5bae7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:13:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bae7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 -- 172.21.15.86:0/1917089639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5baed10 msgr2=0x55bcf5baf670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5baed10 0x55bcf5baf670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.297+0000 7f9eaf400700 1 -- 172.21.15.86:0/1917089639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcf59793b0 con 0x55bcf5bafc00 2024-03-22T19:13:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.298+0000 7f9eaf400700 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb0080 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009f20 tx=0x7f9ea8009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c337c6c478d95b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.298+0000 7f9e9be00700 1 -- 172.21.15.86:0/1917089639 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011070 con 0x55bcf5bafc00 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.298+0000 7f9e9be00700 1 -- 172.21.15.86:0/1917089639 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea8015410 con 0x55bcf5bafc00 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.298+0000 7f9e9be00700 1 -- 172.21.15.86:0/1917089639 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800a660 con 0x55bcf5bafc00 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1917089639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 msgr2=0x55bcf5bb0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb0080 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009f20 tx=0x7f9ea8009f50 comp rx=0 tx=0).stop 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1917089639 shutdown_connections 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb0080 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1917089639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5baed10 0x55bcf5baf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.299+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1917089639 >> 172.21.15.86:0/1917089639 conn(0x55bcf5b1b220 msgr2=0x55bcf5ba0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.303+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1917089639 shutdown_connections 2024-03-22T19:13:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.303+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1917089639 wait complete. 2024-03-22T19:13:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.304+0000 7f9ec943fb80 1 -- start start 2024-03-22T19:13:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.304+0000 7f9ec943fb80 1 Processor -- start 2024-03-22T19:13:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.305+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bb8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.305+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5bbfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9eaea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bb8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9eaea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bb8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50344/0 (socket says 172.21.15.86:50344) 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9eaea00700 1 -- 172.21.15.86:0/4254343723 learned_addr learned my addr 172.21.15.86:0/4254343723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5a93d80 con 0x55bcf5bafc00 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5a93a20 con 0x55bcf5bae3b0 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9eaf400700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5bbfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf597b6a0 con 0x55bcf5bbda00 2024-03-22T19:13:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.306+0000 7f9ec943fb80 10 client.0 _ll_register_callbacks cb 0x55bcf58ff3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eae000700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eaf400700 1 -- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 msgr2=0x55bcf5bb8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eaf400700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bb8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eaf400700 1 -- 172.21.15.86:0/4254343723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 msgr2=0x55bcf5bb89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eaf400700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eaf400700 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea8009bd0 con 0x55bcf5bbda00 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eae000700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9eaf400700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5bbfdf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8004c20 tx=0x7f9ea800c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1a9c033b87557b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9e98c00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9e98c00700 1 -- 172.21.15.86:0/4254343723 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011040 con 0x55bcf5bbda00 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.307+0000 7f9e98c00700 1 -- 172.21.15.86:0/4254343723 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9ea8006e40 con 0x55bcf5bbda00 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.308+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcf5b98ad0 con 0x55bcf5bbda00 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.308+0000 7f9e98c00700 1 -- 172.21.15.86:0/4254343723 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea8020480 con 0x55bcf5bbda00 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.308+0000 7f9ec943fb80 10 client.25954 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.308+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bcf5925b60 con 0x55bcf5bbda00 2024-03-22T19:13:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.308+0000 7f9e98c00700 1 -- 172.21.15.86:0/4254343723 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea802f020 con 0x55bcf5bbda00 2024-03-22T19:13:40.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9e98c00700 1 -- 172.21.15.86:0/4254343723 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ea801c030 con 0x55bcf5bbda00 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9ec943fb80 10 client.25954 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9ec943fb80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9ec943fb80 20 client.25954 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5bc4500 0x55bcf5bc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9ec943fb80 4 client.25954 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcf5af0700 con 0x55bcf5bc4500 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.309+0000 7f9eaea00700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5bc4500 0x55bcf5bc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.310+0000 7f9eaea00700 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5bc4500 0x55bcf5bc6920 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57ef6ef070fc83d1 server_cookie=136013ce43e8bab8 in_seq=0 out_seq=0 2024-03-22T19:13:40.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.310+0000 7f9e98c00700 10 client.25954 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9e98c00700 1 -- 172.21.15.86:0/4254343723 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bcf5af0700 con 0x55bcf5bc4500 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9e98c00700 10 client.25954 handle_command_reply: tid=0 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9ec943fb80 1 client.25954 shutdown 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 msgr2=0x55bcf5bbfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5bbfdf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8004c20 tx=0x7f9ea800c360 comp rx=0 tx=0).stop 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 shutdown_connections 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.311+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5bc4500 0x55bcf5bc6920 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5bbfdf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bb8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/4254343723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5bb89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 >> 172.21.15.86:0/4254343723 conn(0x55bcf5b1b220 msgr2=0x55bcf5bad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9e98c00700 0 client.25954 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 shutdown_connections 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 1 -- 172.21.15.86:0/4254343723 wait complete. 2024-03-22T19:13:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.312+0000 7f9ec943fb80 20 client.25954 trim_cache size 0 max 16384 2024-03-22T19:13:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.346+0000 7f9ec943fb80 1 Processor -- start 2024-03-22T19:13:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.346+0000 7f9ec943fb80 1 -- start start 2024-03-22T19:13:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.347+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bd3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.347+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 0x55bcf5baf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.347+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bbda00 0x55bcf5bb79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.347+0000 7f9ec943fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5af0700 con 0x55bcf5bbda00 2024-03-22T19:13:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.347+0000 7f9ec943fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5b6b940 con 0x55bcf5bae3b0 2024-03-22T19:13:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.347+0000 7f9ec943fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5bc0f80 con 0x55bcf5bafc00 2024-03-22T19:13:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.348+0000 7f9eae000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 0x55bcf5baf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.348+0000 7f9eae000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 0x55bcf5baf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59692/0 (socket says 172.21.15.86:59692) 2024-03-22T19:13:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.348+0000 7f9eae000700 1 -- 172.21.15.86:0/1124016368 learned_addr learned my addr 172.21.15.86:0/1124016368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.348+0000 7f9eaea00700 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bd3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9eae000700 1 -- 172.21.15.86:0/1124016368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 msgr2=0x55bcf5bd3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9eae000700 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bd3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9eae000700 1 -- 172.21.15.86:0/1124016368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bbda00 msgr2=0x55bcf5bb79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9eae000700 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bbda00 0x55bcf5bb79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9eae000700 1 -- 172.21.15.86:0/1124016368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcf5abdea0 con 0x55bcf5bafc00 2024-03-22T19:13:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9eae000700 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 0x55bcf5baf5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c097680 tx=0x7f9e9c09a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6662c15b217f2692 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.349+0000 7f9ea4c00700 1 -- 172.21.15.86:0/1124016368 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c09b310 con 0x55bcf5bafc00 2024-03-22T19:13:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.350+0000 7f9ea4c00700 1 -- 172.21.15.86:0/1124016368 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e9c09ac10 con 0x55bcf5bafc00 2024-03-22T19:13:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.350+0000 7f9ea4c00700 1 -- 172.21.15.86:0/1124016368 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c09b310 con 0x55bcf5bafc00 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.350+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1124016368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 msgr2=0x55bcf5baf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.350+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 0x55bcf5baf5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c097680 tx=0x7f9e9c09a570 comp rx=0 tx=0).stop 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1124016368 shutdown_connections 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bbda00 0x55bcf5bb79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5bd3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/1124016368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bafc00 0x55bcf5baf5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1124016368 >> 172.21.15.86:0/1124016368 conn(0x55bcf5b1b220 msgr2=0x55bcf5b9ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1124016368 shutdown_connections 2024-03-22T19:13:40.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.351+0000 7f9ec943fb80 1 -- 172.21.15.86:0/1124016368 wait complete. 2024-03-22T19:13:40.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.352+0000 7f9ec943fb80 1 -- start start 2024-03-22T19:13:40.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.352+0000 7f9ec943fb80 1 Processor -- start 2024-03-22T19:13:40.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.353+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5c10560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.353+0000 7f9ec943fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5c12ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9eaea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5c10560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9eaea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5c10560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50356/0 (socket says 172.21.15.86:50356) 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9eaea00700 1 -- 172.21.15.86:0/2704639864 learned_addr learned my addr 172.21.15.86:0/2704639864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5c15000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9eae000700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5c12ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5bd40c0 con 0x55bcf5bafc00 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5c15540 con 0x55bcf5bae3b0 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.354+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5c15680 con 0x55bcf5bbda00 2024-03-22T19:13:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5c15000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9ec943fb80 10 client.0 _ll_register_callbacks cb 0x7f9ea8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 -- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 msgr2=0x55bcf5c10560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5c10560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 -- 172.21.15.86:0/2704639864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 msgr2=0x55bcf5c12ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5c12ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb01a12b0 con 0x55bcf5bbda00 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9eaf400700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5c15000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ee30 tx=0x7f9ea800cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3711a0bbf097751 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9e98c00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9e98c00700 1 -- 172.21.15.86:0/2704639864 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011070 con 0x55bcf5bbda00 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.355+0000 7f9e98c00700 1 -- 172.21.15.86:0/2704639864 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9ea8030080 con 0x55bcf5bbda00 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.356+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ea8015410 con 0x55bcf5bbda00 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.356+0000 7f9e98c00700 1 -- 172.21.15.86:0/2704639864 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea8009ea0 con 0x55bcf5bbda00 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.356+0000 7f9ec943fb80 10 client.25960 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:40.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.356+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bcf597dcd0 con 0x55bcf5bbda00 2024-03-22T19:13:40.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.357+0000 7f9e98c00700 1 -- 172.21.15.86:0/2704639864 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea8015410 con 0x55bcf5bbda00 2024-03-22T19:13:40.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9e98c00700 1 -- 172.21.15.86:0/2704639864 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ea801c040 con 0x55bcf5bbda00 2024-03-22T19:13:40.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9ec943fb80 10 client.25960 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:40.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9ec943fb80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:40.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9ec943fb80 20 client.25960 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:40.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5c16cb0 0x55bcf5c190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9ec943fb80 4 client.25960 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:13:40.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bcf5946cf0 con 0x55bcf5c16cb0 2024-03-22T19:13:40.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.358+0000 7f9eaea00700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5c16cb0 0x55bcf5c190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.359+0000 7f9e98c00700 10 client.25960 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:40.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.359+0000 7f9eaea00700 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5c16cb0 0x55bcf5c190a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751a0a6d2cdecc84 server_cookie=115cbad7f86adde6 in_seq=0 out_seq=0 2024-03-22T19:13:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9e98c00700 1 -- 172.21.15.86:0/2704639864 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bcf5946cf0 con 0x55bcf5c16cb0 2024-03-22T19:13:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9e98c00700 10 client.25960 handle_command_reply: tid=0 2024-03-22T19:13:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 client.25960 shutdown 2024-03-22T19:13:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 msgr2=0x55bcf5c15000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5c15000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ee30 tx=0x7f9ea800cf80 comp rx=0 tx=0).stop 2024-03-22T19:13:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 shutdown_connections 2024-03-22T19:13:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55bcf5c16cb0 0x55bcf5c190a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcf5bbda00 0x55bcf5c15000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcf5bae3b0 0x55bcf5c10560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 --2- 172.21.15.86:0/2704639864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcf5bafc00 0x55bcf5c12ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 >> 172.21.15.86:0/2704639864 conn(0x55bcf5b1b220 msgr2=0x55bcf5bb8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.360+0000 7f9e98c00700 0 client.25960 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.362+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 shutdown_connections 2024-03-22T19:13:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.362+0000 7f9ec943fb80 1 -- 172.21.15.86:0/2704639864 wait complete. 2024-03-22T19:13:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.362+0000 7f9ec943fb80 20 client.25960 trim_cache size 0 max 16384 2024-03-22T19:13:40.364 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:13:40.364 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:13:40.364 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:13:40.364 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:13:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.363+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9e940a0b40 msgr2=0x7f9e940a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.363+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9e940a0b40 0x7f9e940a3000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0100c80 tx=0x7f9ea0009260 comp rx=0 tx=0).stop 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.363+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 msgr2=0x7f9eb01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.363+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 0x7f9eb01993b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c003760 tx=0x7f9e9c003f00 comp rx=0 tx=0).stop 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 shutdown_connections 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9e940a0b40 0x7f9e940a3000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb01017c0 0x7f9eb01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb00ffc30 0x7f9eb0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 --2- 172.21.15.86:0/3314435963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb0100e20 0x7f9eb01993b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 >> 172.21.15.86:0/3314435963 conn(0x7f9eb00fb1a0 msgr2=0x7f9eb00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 shutdown_connections 2024-03-22T19:13:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.364+0000 7f9eb5a00700 1 -- 172.21.15.86:0/3314435963 wait complete. 2024-03-22T19:13:40.408 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:13:40.408 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 -- 172.21.15.86:0/2534645330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca00101690 msgr2=0x7fca00103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/2534645330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca00101690 0x7fca00103a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b0a0 tx=0x7fc9f000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 -- 172.21.15.86:0/2534645330 shutdown_connections 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/2534645330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca001068f0 0x7fca00108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/2534645330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca00101690 0x7fca00103a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/2534645330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca00103fc0 0x7fca001063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.875+0000 7fc9ffe00700 1 -- 172.21.15.86:0/2534645330 >> 172.21.15.86:0/2534645330 conn(0x7fca000fb060 msgr2=0x7fca000fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.876+0000 7fc9ffe00700 1 -- 172.21.15.86:0/2534645330 shutdown_connections 2024-03-22T19:13:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.876+0000 7fc9ffe00700 1 -- 172.21.15.86:0/2534645330 wait complete. 2024-03-22T19:13:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.878+0000 7fc9ffe00700 1 Processor -- start 2024-03-22T19:13:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.878+0000 7fc9ffe00700 1 -- start start 2024-03-22T19:13:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.879+0000 7fc9ffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca00101690 0x7fca00194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.879+0000 7fc9ffe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca00103fc0 0x7fca00194fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.879+0000 7fc9ffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 0x7fca0019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ffe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0019c580 con 0x7fca00101690 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ffe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0019c6f0 con 0x7fca001068f0 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ffe00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca001ad870 con 0x7fca00103fc0 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 0x7fca0019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 0x7fca0019bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50366/0 (socket says 172.21.15.86:50366) 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 -- 172.21.15.86:0/1960926969 learned_addr learned my addr 172.21.15.86:0/1960926969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9f7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca00103fc0 0x7fca00194fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 -- 172.21.15.86:0/1960926969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca00103fc0 msgr2=0x7fca00194fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca00103fc0 0x7fca00194fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 -- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca00101690 msgr2=0x7fca00194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca00101690 0x7fca00194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.880+0000 7fc9ff400700 1 -- 172.21.15.86:0/1960926969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f0009d00 con 0x7fca001068f0 2024-03-22T19:13:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.881+0000 7fc9ff400700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 0x7fca0019bfb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b4a0 tx=0x7fc9f800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.882+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8011830 con 0x7fca001068f0 2024-03-22T19:13:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.882+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9f8011e50 con 0x7fca001068f0 2024-03-22T19:13:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.882+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8011830 con 0x7fca001068f0 2024-03-22T19:13:40.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.882+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca001adaf0 con 0x7fca001068f0 2024-03-22T19:13:40.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.882+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca001adfb0 con 0x7fca001068f0 2024-03-22T19:13:40.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.885+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca000fcbe0 con 0x7fca001068f0 2024-03-22T19:13:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.889+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fc9f8010480 con 0x7fca001068f0 2024-03-22T19:13:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.891+0000 7fc9f7400700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc9e40a0ba0 0x7fc9e40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.892+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc9f80d2350 con 0x7fca001068f0 2024-03-22T19:13:40.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.894+0000 7fc9fea00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc9e40a0ba0 0x7fc9e40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.895+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc9f80cf060 con 0x7fca001068f0 2024-03-22T19:13:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:40.896+0000 7fc9fea00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc9e40a0ba0 0x7fc9e40a3060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b070 tx=0x7fc9f0007c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:41.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.486+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca00067970 con 0x7fca001068f0 2024-03-22T19:13:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.487+0000 7fc9f7400700 1 -- 172.21.15.86:0/1960926969 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10934 (secure 0 0 0) 0x7fc9f80943d0 con 0x7fca001068f0 2024-03-22T19:13:41.492 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:13:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:13:36.377933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:13:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.492+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc9e40a0ba0 msgr2=0x7fc9e40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.492+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc9e40a0ba0 0x7fc9e40a3060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b070 tx=0x7fc9f0007c30 comp rx=0 tx=0).stop 2024-03-22T19:13:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.492+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 msgr2=0x7fca0019bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.492+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 0x7fca0019bfb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b4a0 tx=0x7fc9f800b860 comp rx=0 tx=0).stop 2024-03-22T19:13:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 shutdown_connections 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc9e40a0ba0 0x7fc9e40a3060 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca001068f0 0x7fca0019bfb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca00101690 0x7fca00194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 --2- 172.21.15.86:0/1960926969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca00103fc0 0x7fca00194fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 >> 172.21.15.86:0/1960926969 conn(0x7fca000fb060 msgr2=0x7fca00101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.493+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 shutdown_connections 2024-03-22T19:13:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.494+0000 7fc9ffe00700 1 -- 172.21.15.86:0/1960926969 wait complete. 2024-03-22T19:13:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-03-22T19:13:41.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:13:41.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:41 smithi131 ceph-mon[78559]: pgmap v367: 97 pgs: 97 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 175 MiB/s wr, 14.89k op/s 2024-03-22T19:13:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:41 smithi086 ceph-mon[38032]: pgmap v367: 97 pgs: 97 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 175 MiB/s wr, 14.89k op/s 2024-03-22T19:13:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:41 smithi167 ceph-mon[47728]: pgmap v367: 97 pgs: 97 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 175 MiB/s wr, 14.89k op/s 2024-03-22T19:13:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.877+0000 7f1297a00700 1 -- 172.21.15.86:0/3610716712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 msgr2=0x7f129010a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.877+0000 7f128f400700 1 -- 172.21.15.86:0/3610716712 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880092e0 con 0x7f1290107f90 2024-03-22T19:13:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.877+0000 7f1297a00700 1 --2- 172.21.15.86:0/3610716712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f129010a380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f128800b0a0 tx=0x7f128800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.878+0000 7f1297a00700 1 -- 172.21.15.86:0/3610716712 shutdown_connections 2024-03-22T19:13:41.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.878+0000 7f1297a00700 1 --2- 172.21.15.86:0/3610716712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f129010d410 0x7f129010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.878+0000 7f1297a00700 1 --2- 172.21.15.86:0/3610716712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f129010a380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.878+0000 7f1297a00700 1 --2- 172.21.15.86:0/3610716712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f129010a950 0x7f129010cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.878+0000 7f1297a00700 1 -- 172.21.15.86:0/3610716712 >> 172.21.15.86:0/3610716712 conn(0x7f129006df10 msgr2=0x7f1290070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.879+0000 7f1297a00700 1 -- 172.21.15.86:0/3610716712 shutdown_connections 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.880+0000 7f1297a00700 1 -- 172.21.15.86:0/3610716712 wait complete. 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.881+0000 7f1297a00700 1 Processor -- start 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 -- start start 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f12901a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f129010a950 0x7f12901a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f129010d410 0x7f12901acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12901ad570 con 0x7f129010a950 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12901ad6b0 con 0x7f129010d410 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.883+0000 7f1297a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12901ad840 con 0x7f1290107f90 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.885+0000 7f1295000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f12901a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.885+0000 7f1295000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f12901a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59736/0 (socket says 172.21.15.86:59736) 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.885+0000 7f1295000700 1 -- 172.21.15.86:0/2703542404 learned_addr learned my addr 172.21.15.86:0/2703542404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.885+0000 7f128fe00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f129010a950 0x7f12901a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.886+0000 7f1295000700 1 -- 172.21.15.86:0/2703542404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f129010d410 msgr2=0x7f12901acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.886+0000 7f1295000700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f129010d410 0x7f12901acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.886+0000 7f1295000700 1 -- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f129010a950 msgr2=0x7f12901a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.886+0000 7f1295000700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f129010a950 0x7f12901a5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.886+0000 7f1295000700 1 -- 172.21.15.86:0/2703542404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288009d00 con 0x7f1290107f90 2024-03-22T19:13:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.886+0000 7f1295000700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f12901a5a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1288007820 tx=0x7f1288007900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.888+0000 7f128d600700 1 -- 172.21.15.86:0/2703542404 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288003e10 con 0x7f1290107f90 2024-03-22T19:13:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.888+0000 7f128d600700 1 -- 172.21.15.86:0/2703542404 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1288005530 con 0x7f1290107f90 2024-03-22T19:13:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.888+0000 7f128d600700 1 -- 172.21.15.86:0/2703542404 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128801aa60 con 0x7f1290107f90 2024-03-22T19:13:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.888+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12901adac0 con 0x7f1290107f90 2024-03-22T19:13:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.888+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12901ade90 con 0x7f1290107f90 2024-03-22T19:13:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.898+0000 7f128d600700 1 -- 172.21.15.86:0/2703542404 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f128801abc0 con 0x7f1290107f90 2024-03-22T19:13:41.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.903+0000 7f128d600700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0c10 0x7f12740a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.904+0000 7f128d600700 1 -- 172.21.15.86:0/2703542404 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f12880ce1d0 con 0x7f1290107f90 2024-03-22T19:13:41.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.904+0000 7f128fe00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0c10 0x7f12740a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.913+0000 7f128fe00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0c10 0x7f12740a30d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1290072fb0 tx=0x7f1280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.918+0000 7f12ab5a2b80 1 Processor -- start 2024-03-22T19:13:41.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.922+0000 7f12ab5a2b80 1 -- start start 2024-03-22T19:13:41.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.924+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa285d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.924+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 0x55faa285de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.924+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285e420 0x55faa285e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.924+0000 7f12ab5a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa27b5990 con 0x55faa285e420 2024-03-22T19:13:41.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.924+0000 7f12ab5a2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa27b3350 con 0x55faa285d580 2024-03-22T19:13:41.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.924+0000 7f12ab5a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa27aec40 con 0x55faa285cc20 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.925+0000 7f128fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 0x55faa285de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.925+0000 7f128fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 0x55faa285de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50386/0 (socket says 172.21.15.86:50386) 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.925+0000 7f128fe00700 1 -- 172.21.15.86:0/66864623 learned_addr learned my addr 172.21.15.86:0/66864623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.925+0000 7f1295a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285e420 0x55faa285e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.926+0000 7f128fe00700 1 -- 172.21.15.86:0/66864623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 msgr2=0x55faa285d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.926+0000 7f128fe00700 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa285d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.926+0000 7f128fe00700 1 -- 172.21.15.86:0/66864623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285e420 msgr2=0x55faa285e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.926+0000 7f128fe00700 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285e420 0x55faa285e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.926+0000 7f128fe00700 1 -- 172.21.15.86:0/66864623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faa26303b0 con 0x55faa285d580 2024-03-22T19:13:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.929+0000 7f128fe00700 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 0x55faa285de40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1280011e10 tx=0x7f1280014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d105f5d279f49b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.930+0000 7f1286000700 1 -- 172.21.15.86:0/66864623 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128001f420 con 0x55faa285d580 2024-03-22T19:13:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.932+0000 7f1286000700 1 -- 172.21.15.86:0/66864623 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1280016070 con 0x55faa285d580 2024-03-22T19:13:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.933+0000 7f1286000700 1 -- 172.21.15.86:0/66864623 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128001d920 con 0x55faa285d580 2024-03-22T19:13:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/66864623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 msgr2=0x55faa285de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 0x55faa285de40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1280011e10 tx=0x7f1280014000 comp rx=0 tx=0).stop 2024-03-22T19:13:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/66864623 shutdown_connections 2024-03-22T19:13:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285e420 0x55faa285e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa285d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/66864623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285d580 0x55faa285de40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.934+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/66864623 >> 172.21.15.86:0/66864623 conn(0x55faa27d2220 msgr2=0x55faa2857bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.936+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/66864623 shutdown_connections 2024-03-22T19:13:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.936+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/66864623 wait complete. 2024-03-22T19:13:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.937+0000 7f12ab5a2b80 1 -- start start 2024-03-22T19:13:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.939+0000 7f12ab5a2b80 1 Processor -- start 2024-03-22T19:13:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.939+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa2866b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.940+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa2867060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.940+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.941+0000 7f128fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa2867060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.941+0000 7f128fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa2867060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38768/0 (socket says 172.21.15.86:38768) 2024-03-22T19:13:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.941+0000 7f1295000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa2866b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.941+0000 7f1295000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa2866b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59756/0 (socket says 172.21.15.86:59756) 2024-03-22T19:13:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.942+0000 7f128fe00700 1 -- 172.21.15.86:0/1013037822 learned_addr learned my addr 172.21.15.86:0/1013037822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.942+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa274ad80 con 0x55faa285d580 2024-03-22T19:13:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.942+0000 7f1295a00700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.942+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa274aa20 con 0x55faa285e420 2024-03-22T19:13:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.942+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa26326a0 con 0x55faa285cc20 2024-03-22T19:13:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.943+0000 7f1295000700 1 -- 172.21.15.86:0/1013037822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 msgr2=0x55faa286e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.943+0000 7f1295000700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.943+0000 7f1295000700 1 -- 172.21.15.86:0/1013037822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 msgr2=0x55faa2867060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.944+0000 7f1295000700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa2867060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.944+0000 7f1295000700 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1280011af0 con 0x55faa285cc20 2024-03-22T19:13:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.944+0000 7f128fe00700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa2867060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.945+0000 7f12ab5a2b80 10 client.0 _ll_register_callbacks cb 0x55faa25b63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.945+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faa284fad0 con 0x55faa285cc20 2024-03-22T19:13:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.945+0000 7f1295000700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa2866b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12880cbdb0 tx=0x7f128809b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2369f2a073c8eb85 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.945+0000 7f127aa00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.946+0000 7f127aa00700 1 -- 172.21.15.86:0/1013037822 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128809b980 con 0x55faa285cc20 2024-03-22T19:13:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.946+0000 7f127aa00700 1 -- 172.21.15.86:0/1013037822 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f128809f7b0 con 0x55faa285cc20 2024-03-22T19:13:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.947+0000 7f127aa00700 1 -- 172.21.15.86:0/1013037822 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f128809b980 con 0x55faa285cc20 2024-03-22T19:13:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.947+0000 7f12ab5a2b80 10 client.25972 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.947+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x55faa25dcb60 con 0x55faa285cc20 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.949+0000 7f127aa00700 1 -- 172.21.15.86:0/1013037822 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f128809bbe0 con 0x55faa285cc20 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.949+0000 7f12ab5a2b80 10 client.25972 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.949+0000 7f12ab5a2b80 10 client.25972 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.949+0000 7f12ab5a2b80 20 client.25972 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.949+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28721b0 0x55faa2874670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.950+0000 7f12ab5a2b80 4 client.25972 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.950+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faa27a7700 con 0x55faa28721b0 2024-03-22T19:13:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.954+0000 7f128fe00700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28721b0 0x55faa2874670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.955+0000 7f128fe00700 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28721b0 0x55faa2874670 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4efbef85c08d20f server_cookie=10df43561ffecbd1 in_seq=0 out_seq=0 2024-03-22T19:13:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.955+0000 7f127aa00700 10 client.25972 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.958+0000 7f127aa00700 1 -- 172.21.15.86:0/1013037822 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55faa27a7700 con 0x55faa28721b0 2024-03-22T19:13:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.958+0000 7f127aa00700 10 client.25972 handle_command_reply: tid=0 2024-03-22T19:13:41.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.958+0000 7f12ab5a2b80 1 client.25972 shutdown 2024-03-22T19:13:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.960+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 msgr2=0x55faa2866b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.960+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa2866b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12880cbdb0 tx=0x7f128809b820 comp rx=0 tx=0).stop 2024-03-22T19:13:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 shutdown_connections 2024-03-22T19:13:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28721b0 0x55faa2874670 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa2866b20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/1013037822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa2867060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f127aa00700 0 client.25972 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.961+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 >> 172.21.15.86:0/1013037822 conn(0x55faa27d2220 msgr2=0x55faa28571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:41.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.962+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 shutdown_connections 2024-03-22T19:13:41.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.963+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/1013037822 wait complete. 2024-03-22T19:13:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:41.963+0000 7f12ab5a2b80 20 client.25972 trim_cache size 0 max 16384 2024-03-22T19:13:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.011+0000 7f12ab5a2b80 1 Processor -- start 2024-03-22T19:13:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.011+0000 7f12ab5a2b80 1 -- start start 2024-03-22T19:13:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.011+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285cc20 0x55faa285e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.011+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 0x55faa286e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.011+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa2857ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.012+0000 7f12ab5a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa28545e0 con 0x55faa285cc20 2024-03-22T19:13:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.012+0000 7f12ab5a2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa2851f80 con 0x55faa285e420 2024-03-22T19:13:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.012+0000 7f12ab5a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa28793d0 con 0x55faa285d580 2024-03-22T19:13:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.013+0000 7f128fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 0x55faa286e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.013+0000 7f128fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 0x55faa286e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59772/0 (socket says 172.21.15.86:59772) 2024-03-22T19:13:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.013+0000 7f128fe00700 1 -- 172.21.15.86:0/2431938047 learned_addr learned my addr 172.21.15.86:0/2431938047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.013+0000 7f1295000700 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285cc20 0x55faa285e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.014+0000 7f128fe00700 1 -- 172.21.15.86:0/2431938047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 msgr2=0x55faa2857ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.014+0000 7f128fe00700 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa2857ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.014+0000 7f128fe00700 1 -- 172.21.15.86:0/2431938047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285cc20 msgr2=0x55faa285e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.014+0000 7f128fe00700 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285cc20 0x55faa285e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.014+0000 7f128fe00700 1 -- 172.21.15.86:0/2431938047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faa2774ea0 con 0x55faa285d580 2024-03-22T19:13:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.015+0000 7f128fe00700 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 0x55faa286e060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f128001d3f0 tx=0x7f1280017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42fa902394312010 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.016+0000 7f1286a00700 1 -- 172.21.15.86:0/2431938047 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800362d0 con 0x55faa285d580 2024-03-22T19:13:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.017+0000 7f1286a00700 1 -- 172.21.15.86:0/2431938047 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128002f040 con 0x55faa285d580 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.017+0000 7f1286a00700 1 -- 172.21.15.86:0/2431938047 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800050c0 con 0x55faa285d580 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.019+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/2431938047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 msgr2=0x55faa286e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.019+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 0x55faa286e060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f128001d3f0 tx=0x7f1280017000 comp rx=0 tx=0).stop 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.020+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/2431938047 shutdown_connections 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.020+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa2857ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.020+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285cc20 0x55faa285e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.020+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/2431938047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285d580 0x55faa286e060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.020+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/2431938047 >> 172.21.15.86:0/2431938047 conn(0x55faa27d2220 msgr2=0x55faa2866030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.020+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/2431938047 shutdown_connections 2024-03-22T19:13:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.021+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/2431938047 wait complete. 2024-03-22T19:13:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.021+0000 7f12ab5a2b80 1 -- start start 2024-03-22T19:13:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.022+0000 7f12ab5a2b80 1 Processor -- start 2024-03-22T19:13:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.023+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa286a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.023+0000 7f12ab5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa286a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.024+0000 7f1295000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa286a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.024+0000 7f1295000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa286a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59786/0 (socket says 172.21.15.86:59786) 2024-03-22T19:13:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.024+0000 7f1295000700 1 -- 172.21.15.86:0/3285786937 learned_addr learned my addr 172.21.15.86:0/3285786937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.025+0000 7f128fe00700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa286a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.026+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.027+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa286dbf0 con 0x55faa285d580 2024-03-22T19:13:42.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.027+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa286dd60 con 0x55faa285e420 2024-03-22T19:13:42.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.027+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa286df20 con 0x55faa285cc20 2024-03-22T19:13:42.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.027+0000 7f12ab5a2b80 10 client.0 _ll_register_callbacks cb 0x55faa25b63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.027+0000 7f1295a00700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.028+0000 7f1295000700 1 -- 172.21.15.86:0/3285786937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 msgr2=0x55faa286d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.028+0000 7f1295000700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.028+0000 7f1295000700 1 -- 172.21.15.86:0/3285786937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 msgr2=0x55faa286a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.028+0000 7f1295000700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa286a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.028+0000 7f1295000700 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55faa26303b0 con 0x55faa285cc20 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.030+0000 7f127aa00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.030+0000 7f1295000700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa286a240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12880af7e0 tx=0x7f128800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ed27ba47e3fda8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.031+0000 7f127aa00700 1 -- 172.21.15.86:0/3285786937 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128809b2e0 con 0x55faa285cc20 2024-03-22T19:13:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.031+0000 7f127aa00700 1 -- 172.21.15.86:0/3285786937 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f128809f7b0 con 0x55faa285cc20 2024-03-22T19:13:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.031+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1280016070 con 0x55faa285cc20 2024-03-22T19:13:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.031+0000 7f127aa00700 1 -- 172.21.15.86:0/3285786937 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f128809ae70 con 0x55faa285cc20 2024-03-22T19:13:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.032+0000 7f12ab5a2b80 10 client.25978 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.032+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55faa2634cd0 con 0x55faa285cc20 2024-03-22T19:13:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.033+0000 7f127aa00700 1 -- 172.21.15.86:0/3285786937 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1280016070 con 0x55faa285cc20 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f127aa00700 1 -- 172.21.15.86:0/3285786937 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f12880050a0 con 0x55faa285cc20 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f12ab5a2b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f12ab5a2b80 10 client.25978 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f12ab5a2b80 20 client.25978 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28c49e0 0x55faa28c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f12ab5a2b80 4 client.25978 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.037+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55faa25fdcf0 con 0x55faa28c49e0 2024-03-22T19:13:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.038+0000 7f128fe00700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28c49e0 0x55faa28c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.044+0000 7f128fe00700 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28c49e0 0x55faa28c6dd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c0bf20af8515faa server_cookie=ee135ac7390fd8dd in_seq=0 out_seq=0 2024-03-22T19:13:42.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.044+0000 7f127aa00700 10 client.25978 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:42.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.047+0000 7f127aa00700 1 -- 172.21.15.86:0/3285786937 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55faa25fdcf0 con 0x55faa28c49e0 2024-03-22T19:13:42.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.047+0000 7f127aa00700 10 client.25978 handle_command_reply: tid=0 2024-03-22T19:13:42.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.047+0000 7f12ab5a2b80 1 client.25978 shutdown 2024-03-22T19:13:42.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.048+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 msgr2=0x55faa286a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.048+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa286a240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12880af7e0 tx=0x7f128800bf60 comp rx=0 tx=0).stop 2024-03-22T19:13:42.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 shutdown_connections 2024-03-22T19:13:42.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55faa285e420 0x55faa286d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x55faa28c49e0 0x55faa28c6dd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa285cc20 0x55faa286a240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f12ab5a2b80 1 --2- 172.21.15.86:0/3285786937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faa285d580 0x55faa286a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 >> 172.21.15.86:0/3285786937 conn(0x55faa27d2220 msgr2=0x55faa2865f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.049+0000 7f127aa00700 0 client.25978 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:13:42.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.053+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 shutdown_connections 2024-03-22T19:13:42.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.054+0000 7f12ab5a2b80 1 -- 172.21.15.86:0/3285786937 wait complete. 2024-03-22T19:13:42.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.054+0000 7f12ab5a2b80 20 client.25978 trim_cache size 0 max 16384 2024-03-22T19:13:42.056 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:42.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0c10 msgr2=0x7f12740a30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0c10 0x7f12740a30d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1290072fb0 tx=0x7f1280009260 comp rx=0 tx=0).stop 2024-03-22T19:13:42.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 msgr2=0x7f12901a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f12901a5a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1288007820 tx=0x7f1288007900 comp rx=0 tx=0).stop 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 shutdown_connections 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f12740a0c10 0x7f12740a30d0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1290072fb0 tx=0x7f1280009260 comp rx=0 tx=0).stop 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f129010d410 0x7f12901acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290107f90 0x7f12901a5a40 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1288007820 tx=0x7f1288007900 comp rx=0 tx=0).stop 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 --2- 172.21.15.86:0/2703542404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f129010a950 0x7f12901a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 >> 172.21.15.86:0/2703542404 conn(0x7f129006df10 msgr2=0x7f129010b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.056+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 shutdown_connections 2024-03-22T19:13:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.057+0000 7f1297a00700 1 -- 172.21.15.86:0/2703542404 wait complete. 2024-03-22T19:13:42.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:13:42.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:42 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1960926969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 -- 172.21.15.86:0/2801022891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108540 msgr2=0x7ff8901089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 --2- 172.21.15.86:0/2801022891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108540 0x7ff8901089a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099c0 tx=0x7ff87c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 -- 172.21.15.86:0/2801022891 shutdown_connections 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 --2- 172.21.15.86:0/2801022891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108ee0 0x7ff89010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 --2- 172.21.15.86:0/2801022891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 0x7ff890107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 --2- 172.21.15.86:0/2801022891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108540 0x7ff8901089a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.621+0000 7ff895600700 1 -- 172.21.15.86:0/2801022891 >> 172.21.15.86:0/2801022891 conn(0x7ff8900762e0 msgr2=0x7ff890078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.622+0000 7ff895600700 1 -- 172.21.15.86:0/2801022891 shutdown_connections 2024-03-22T19:13:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.622+0000 7ff895600700 1 -- 172.21.15.86:0/2801022891 wait complete. 2024-03-22T19:13:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.624+0000 7ff895600700 1 Processor -- start 2024-03-22T19:13:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.624+0000 7ff895600700 1 -- start start 2024-03-22T19:13:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff895600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 0x7ff89019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff895600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108540 0x7ff89019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff895600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108ee0 0x7ff8901a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff895600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901a4c30 con 0x7ff890107350 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff895600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890079600 con 0x7ff890108ee0 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff895600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890079790 con 0x7ff890108540 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff88e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108540 0x7ff89019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff88e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108540 0x7ff89019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59806/0 (socket says 172.21.15.86:59806) 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.625+0000 7ff88e000700 1 -- 172.21.15.86:0/3689505533 learned_addr learned my addr 172.21.15.86:0/3689505533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.626+0000 7ff88ea00700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 0x7ff89019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.627+0000 7ff88ea00700 1 -- 172.21.15.86:0/3689505533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108540 msgr2=0x7ff89019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.628+0000 7ff88f400700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108ee0 0x7ff8901a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.628+0000 7ff88ea00700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108540 0x7ff89019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.628+0000 7ff88ea00700 1 -- 172.21.15.86:0/3689505533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108ee0 msgr2=0x7ff8901a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.628+0000 7ff88ea00700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108ee0 0x7ff8901a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.628+0000 7ff88ea00700 1 -- 172.21.15.86:0/3689505533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0095f0 con 0x7ff890107350 2024-03-22T19:13:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.629+0000 7ff88f400700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108ee0 0x7ff8901a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.629+0000 7ff88ea00700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 0x7ff89019d100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e9c0 tx=0x7ff88000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.630+0000 7ff887400700 1 -- 172.21.15.86:0/3689505533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000cbd0 con 0x7ff890107350 2024-03-22T19:13:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.630+0000 7ff887400700 1 -- 172.21.15.86:0/3689505533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8800044e0 con 0x7ff890107350 2024-03-22T19:13:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.630+0000 7ff887400700 1 -- 172.21.15.86:0/3689505533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880003d80 con 0x7ff890107350 2024-03-22T19:13:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.631+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff890079a70 con 0x7ff890107350 2024-03-22T19:13:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.631+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff890079f30 con 0x7ff890107350 2024-03-22T19:13:42.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.641+0000 7ff8a9038b80 1 Processor -- start 2024-03-22T19:13:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.642+0000 7ff8a9038b80 1 -- start start 2024-03-22T19:13:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.645+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x55676786d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.645+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676786de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.645+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676786e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.645+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567677c5990 con 0x55676786d580 2024-03-22T19:13:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.645+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567677c3350 con 0x55676786cc20 2024-03-22T19:13:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.645+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567677bec40 con 0x55676786e420 2024-03-22T19:13:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.646+0000 7ff88e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676786e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.646+0000 7ff88e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676786e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59810/0 (socket says 172.21.15.86:59810) 2024-03-22T19:13:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.646+0000 7ff88e000700 1 -- 172.21.15.86:0/2697883206 learned_addr learned my addr 172.21.15.86:0/2697883206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.646+0000 7ff88f400700 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x55676786d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.646+0000 7ff887400700 1 -- 172.21.15.86:0/3689505533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff880003ee0 con 0x7ff890107350 2024-03-22T19:13:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.647+0000 7ff88f400700 1 -- 172.21.15.86:0/2697883206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 msgr2=0x55676786e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.647+0000 7ff88f400700 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676786e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.648+0000 7ff88f400700 1 -- 172.21.15.86:0/2697883206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 msgr2=0x55676786de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.648+0000 7ff88ea00700 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676786de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.648+0000 7ff88f400700 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676786de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.648+0000 7ff88f400700 1 -- 172.21.15.86:0/2697883206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567676403b0 con 0x55676786cc20 2024-03-22T19:13:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.648+0000 7ff88ea00700 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676786de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.649+0000 7ff887400700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8740a0b60 0x7ff8740a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.649+0000 7ff88f400700 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x55676786d040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff8880084a0 tx=0x7ff888007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c55cced18acadd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.649+0000 7ff887400700 1 -- 172.21.15.86:0/3689505533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff880014070 con 0x7ff890107350 2024-03-22T19:13:42.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.650+0000 7ff87b400700 1 -- 172.21.15.86:0/2697883206 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888011710 con 0x55676786cc20 2024-03-22T19:13:42.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.650+0000 7ff87b400700 1 -- 172.21.15.86:0/2697883206 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff888008e70 con 0x55676786cc20 2024-03-22T19:13:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.650+0000 7ff87b400700 1 -- 172.21.15.86:0/2697883206 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888013c50 con 0x55676786cc20 2024-03-22T19:13:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.650+0000 7ff88ea00700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8740a0b60 0x7ff8740a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2697883206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 msgr2=0x55676786d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x55676786d040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff8880084a0 tx=0x7ff888007f90 comp rx=0 tx=0).stop 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff88ea00700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8740a0b60 0x7ff8740a3020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8800996e0 tx=0x7ff880003820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2697883206 shutdown_connections 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676786e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x55676786d040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2697883206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676786de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.651+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2697883206 >> 172.21.15.86:0/2697883206 conn(0x5567677e2220 msgr2=0x556767867bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.652+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2697883206 shutdown_connections 2024-03-22T19:13:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.652+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2697883206 wait complete. 2024-03-22T19:13:42.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.652+0000 7ff8a9038b80 1 -- start start 2024-03-22T19:13:42.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.653+0000 7ff8a9038b80 1 Processor -- start 2024-03-22T19:13:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x556767876b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x5567678770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676787e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676775ad80 con 0x55676786d580 2024-03-22T19:13:42.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676775aa20 con 0x55676786cc20 2024-03-22T19:13:42.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567676426a0 con 0x55676786e420 2024-03-22T19:13:42.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.655+0000 7ff8a9038b80 10 client.0 _ll_register_callbacks cb 0x5567675c63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.656+0000 7ff88f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x556767876b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.656+0000 7ff88f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x556767876b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50452/0 (socket says 172.21.15.86:50452) 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.656+0000 7ff88f400700 1 -- 172.21.15.86:0/1723442949 learned_addr learned my addr 172.21.15.86:0/1723442949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.656+0000 7ff88f400700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676787e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.656+0000 7ff88e000700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x5567678770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff88f400700 1 -- 172.21.15.86:0/1723442949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 msgr2=0x55676787e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff88f400700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676787e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff88f400700 1 -- 172.21.15.86:0/1723442949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 msgr2=0x5567678770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff88f400700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x5567678770c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff88f400700 1 -- 172.21.15.86:0/1723442949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff888007c70 con 0x55676786cc20 2024-03-22T19:13:42.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff88f400700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x556767876b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff888013550 tx=0x7ff888004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463679f6ac29b35f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.657+0000 7ff86fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.659+0000 7ff86fe00700 1 -- 172.21.15.86:0/1723442949 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888011c50 con 0x55676786cc20 2024-03-22T19:13:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.659+0000 7ff86fe00700 1 -- 172.21.15.86:0/1723442949 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff888011db0 con 0x55676786cc20 2024-03-22T19:13:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.660+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55676785fad0 con 0x55676786cc20 2024-03-22T19:13:42.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.661+0000 7ff86fe00700 1 -- 172.21.15.86:0/1723442949 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff88801c460 con 0x55676786cc20 2024-03-22T19:13:42.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.661+0000 7ff8a9038b80 10 client.26000 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:42.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.661+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5567675ecb60 con 0x55676786cc20 2024-03-22T19:13:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.661+0000 7ff86fe00700 1 -- 172.21.15.86:0/1723442949 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55676785fad0 con 0x55676786cc20 2024-03-22T19:13:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff86fe00700 1 -- 172.21.15.86:0/1723442949 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff8880084d0 con 0x55676786cc20 2024-03-22T19:13:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff8a9038b80 10 client.26000 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff8a9038b80 10 client.26000 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:13:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff8a9038b80 20 client.26000 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x556767895a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff8a9038b80 4 client.26000 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:42.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.662+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567677b7700 con 0x5567678935e0 2024-03-22T19:13:42.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.663+0000 7ff88e000700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x556767895a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.664+0000 7ff88e000700 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x556767895a00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=789db526114a92f3 server_cookie=85aee08b21765dd9 in_seq=0 out_seq=0 2024-03-22T19:13:42.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.664+0000 7ff86fe00700 10 client.26000 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:42.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.665+0000 7ff86fe00700 1 -- 172.21.15.86:0/1723442949 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5567677b7700 con 0x5567678935e0 2024-03-22T19:13:42.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.665+0000 7ff86fe00700 10 client.26000 handle_command_reply: tid=0 2024-03-22T19:13:42.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.665+0000 7ff8a9038b80 1 client.26000 shutdown 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 msgr2=0x556767876b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x556767876b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff888013550 tx=0x7ff888004a90 comp rx=0 tx=0).stop 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 shutdown_connections 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x556767895a00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786e420 0x55676787e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff86fe00700 0 client.26000 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786cc20 0x556767876b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/1723442949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x5567678770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.666+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 >> 172.21.15.86:0/1723442949 conn(0x5567677e2220 msgr2=0x5567678671f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.667+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 shutdown_connections 2024-03-22T19:13:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.667+0000 7ff8a9038b80 1 -- 172.21.15.86:0/1723442949 wait complete. 2024-03-22T19:13:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.667+0000 7ff8a9038b80 20 client.26000 trim_cache size 0 max 16384 2024-03-22T19:13:42.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.698+0000 7ff8a9038b80 1 Processor -- start 2024-03-22T19:13:42.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.698+0000 7ff8a9038b80 1 -- start start 2024-03-22T19:13:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.699+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 0x55676788f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.699+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676787e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.699+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676786e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.699+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556767877cd0 con 0x55676786d580 2024-03-22T19:13:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.699+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556767868cf0 con 0x55676786e420 2024-03-22T19:13:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.699+0000 7ff8a9038b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676785f610 con 0x55676786cc20 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.700+0000 7ff88f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 0x55676788f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.700+0000 7ff88f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 0x55676788f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59828/0 (socket says 172.21.15.86:59828) 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.700+0000 7ff88f400700 1 -- 172.21.15.86:0/398205119 learned_addr learned my addr 172.21.15.86:0/398205119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.700+0000 7ff88f400700 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676786e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.700+0000 7ff88f400700 1 -- 172.21.15.86:0/398205119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 msgr2=0x55676786e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.700+0000 7ff88f400700 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676786e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.701+0000 7ff88f400700 1 -- 172.21.15.86:0/398205119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 msgr2=0x55676787e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.701+0000 7ff88e000700 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676787e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.701+0000 7ff88f400700 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676787e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.701+0000 7ff88f400700 1 -- 172.21.15.86:0/398205119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556767784ea0 con 0x55676786cc20 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.701+0000 7ff88e000700 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676787e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.701+0000 7ff88f400700 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 0x55676788f720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff888013580 tx=0x7ff888013ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8fcd7a43b71fbef server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.702+0000 7ff87be00700 1 -- 172.21.15.86:0/398205119 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880137c0 con 0x55676786cc20 2024-03-22T19:13:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.702+0000 7ff87be00700 1 -- 172.21.15.86:0/398205119 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88800c070 con 0x55676786cc20 2024-03-22T19:13:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.702+0000 7ff87be00700 1 -- 172.21.15.86:0/398205119 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880137c0 con 0x55676786cc20 2024-03-22T19:13:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 -- 172.21.15.86:0/398205119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 msgr2=0x55676788f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 0x55676788f720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff888013580 tx=0x7ff888013ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 -- 172.21.15.86:0/398205119 shutdown_connections 2024-03-22T19:13:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676786e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786cc20 0x55676788f720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/398205119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786d580 0x55676787e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.703+0000 7ff8a9038b80 1 -- 172.21.15.86:0/398205119 >> 172.21.15.86:0/398205119 conn(0x5567677e2220 msgr2=0x556767863a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.704+0000 7ff8a9038b80 1 -- 172.21.15.86:0/398205119 shutdown_connections 2024-03-22T19:13:42.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.705+0000 7ff8a9038b80 1 -- 172.21.15.86:0/398205119 wait complete. 2024-03-22T19:13:42.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.705+0000 7ff8a9038b80 1 -- start start 2024-03-22T19:13:42.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.706+0000 7ff8a9038b80 1 Processor -- start 2024-03-22T19:13:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.707+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786cc20 0x55676787bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.708+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 0x55676787c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff8a9038b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676787d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff88e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 0x55676787c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff88e000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 0x55676787c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59842/0 (socket says 172.21.15.86:59842) 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff88f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786cc20 0x55676787bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676787d9c0 con 0x55676786cc20 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff88e000700 1 -- 172.21.15.86:0/2378294383 learned_addr learned my addr 172.21.15.86:0/2378294383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676787db30 con 0x55676786e420 2024-03-22T19:13:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676787dcf0 con 0x55676786d580 2024-03-22T19:13:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff8a9038b80 10 client.0 _ll_register_callbacks cb 0x7ff888007e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.709+0000 7ff88f400700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676787d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.710+0000 7ff88f400700 1 -- 172.21.15.86:0/2378294383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 msgr2=0x55676787c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.711+0000 7ff88f400700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 0x55676787c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.711+0000 7ff88f400700 1 -- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 msgr2=0x55676787d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.711+0000 7ff88f400700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676787d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.711+0000 7ff88f400700 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567675ecb60 con 0x55676786cc20 2024-03-22T19:13:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.711+0000 7ff88e000700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 0x55676787c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.712+0000 7ff88f400700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786cc20 0x55676787bf80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff888008b40 tx=0x7ff888007450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7323201041e408 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.714+0000 7ff86fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.715+0000 7ff86fe00700 1 -- 172.21.15.86:0/2378294383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888022d90 con 0x55676786cc20 2024-03-22T19:13:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.715+0000 7ff86fe00700 1 -- 172.21.15.86:0/2378294383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff888011a50 con 0x55676786cc20 2024-03-22T19:13:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.715+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff888008e70 con 0x55676786cc20 2024-03-22T19:13:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.716+0000 7ff86fe00700 1 -- 172.21.15.86:0/2378294383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff888004bd0 con 0x55676786cc20 2024-03-22T19:13:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.716+0000 7ff8a9038b80 10 client.15933 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.716+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556767644cd0 con 0x55676786cc20 2024-03-22T19:13:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff86fe00700 1 -- 172.21.15.86:0/2378294383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff888008e70 con 0x55676786cc20 2024-03-22T19:13:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff86fe00700 1 -- 172.21.15.86:0/2378294383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff888011c10 con 0x55676786cc20 2024-03-22T19:13:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff8a9038b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff8a9038b80 10 client.15933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:13:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff8a9038b80 20 client.15933 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x5567678e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff8a9038b80 4 client.15933 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.717+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55676760dcf0 con 0x5567678935e0 2024-03-22T19:13:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.718+0000 7ff88e000700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x5567678e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.719+0000 7ff86fe00700 10 client.15933 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.719+0000 7ff88e000700 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x5567678e7d80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ced5f9cbc8b0fd9f server_cookie=eafc93b9ac9e634b in_seq=0 out_seq=0 2024-03-22T19:13:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.720+0000 7ff86fe00700 1 -- 172.21.15.86:0/2378294383 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55676760dcf0 con 0x5567678935e0 2024-03-22T19:13:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.720+0000 7ff86fe00700 10 client.15933 handle_command_reply: tid=0 2024-03-22T19:13:42.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:42 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1960926969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:13:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.720+0000 7ff8a9038b80 1 client.15933 shutdown 2024-03-22T19:13:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.720+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786cc20 msgr2=0x55676787bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.720+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786cc20 0x55676787bf80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff888008b40 tx=0x7ff888007450 comp rx=0 tx=0).stop 2024-03-22T19:13:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 shutdown_connections 2024-03-22T19:13:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55676786e420 0x55676787d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55676786cc20 0x55676787bf80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5567678935e0 0x5567678e7d80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 --2- 172.21.15.86:0/2378294383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55676786d580 0x55676787c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 >> 172.21.15.86:0/2378294383 conn(0x5567677e2220 msgr2=0x556767876090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff86fe00700 0 client.15933 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:13:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.721+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 shutdown_connections 2024-03-22T19:13:42.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.723+0000 7ff8a9038b80 1 -- 172.21.15.86:0/2378294383 wait complete. 2024-03-22T19:13:42.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.723+0000 7ff8a9038b80 20 client.15933 trim_cache size 0 max 16384 2024-03-22T19:13:42.724 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8740a0b60 msgr2=0x7ff8740a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8740a0b60 0x7ff8740a3020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8800996e0 tx=0x7ff880003820 comp rx=0 tx=0).stop 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 msgr2=0x7ff89019d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 0x7ff89019d100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e9c0 tx=0x7ff88000ed80 comp rx=0 tx=0).stop 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 shutdown_connections 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff8740a0b60 0x7ff8740a3020 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff890108ee0 0x7ff8901a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff890107350 0x7ff89019d100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 --2- 172.21.15.86:0/3689505533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff890108540 0x7ff89019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.724+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 >> 172.21.15.86:0/3689505533 conn(0x7ff8900762e0 msgr2=0x7ff89010b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.725+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 shutdown_connections 2024-03-22T19:13:42.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:42.725+0000 7ff895600700 1 -- 172.21.15.86:0/3689505533 wait complete. 2024-03-22T19:13:42.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:13:42.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:42 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1960926969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:13:43.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.137+0000 7ff37b400700 1 -- 172.21.15.86:0/432475326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 msgr2=0x7ff374103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.137+0000 7ff37b400700 1 --2- 172.21.15.86:0/432475326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 0x7ff374103c60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff368009990 tx=0x7ff368009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.139+0000 7ff37b400700 1 -- 172.21.15.86:0/432475326 shutdown_connections 2024-03-22T19:13:43.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.139+0000 7ff37b400700 1 --2- 172.21.15.86:0/432475326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 0x7ff374108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.139+0000 7ff37b400700 1 --2- 172.21.15.86:0/432475326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 0x7ff374103c60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.139+0000 7ff37b400700 1 --2- 172.21.15.86:0/432475326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 0x7ff374106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.139+0000 7ff37b400700 1 -- 172.21.15.86:0/432475326 >> 172.21.15.86:0/432475326 conn(0x7ff3740fb1a0 msgr2=0x7ff3740fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.140+0000 7ff37b400700 1 -- 172.21.15.86:0/432475326 shutdown_connections 2024-03-22T19:13:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.140+0000 7ff37b400700 1 -- 172.21.15.86:0/432475326 wait complete. 2024-03-22T19:13:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.141+0000 7ff37b400700 1 Processor -- start 2024-03-22T19:13:43.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.142+0000 7ff37b400700 1 -- start start 2024-03-22T19:13:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.143+0000 7ff37b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 0x7ff37406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.143+0000 7ff37b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 0x7ff37406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.143+0000 7ff37b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 0x7ff3741a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.143+0000 7ff378a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 0x7ff37406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.143+0000 7ff373e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 0x7ff37406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.143+0000 7ff373e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 0x7ff37406fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50480/0 (socket says 172.21.15.86:50480) 2024-03-22T19:13:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.144+0000 7ff373e00700 1 -- 172.21.15.86:0/1095216883 learned_addr learned my addr 172.21.15.86:0/1095216883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.144+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374072c80 con 0x7ff374106ad0 2024-03-22T19:13:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.144+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374072df0 con 0x7ff3741041a0 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.144+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3741a7330 con 0x7ff374101870 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.144+0000 7ff379400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 0x7ff3741a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.145+0000 7ff379400700 1 -- 172.21.15.86:0/1095216883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 msgr2=0x7ff37406f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.145+0000 7ff379400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 0x7ff37406f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.145+0000 7ff379400700 1 -- 172.21.15.86:0/1095216883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 msgr2=0x7ff37406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.145+0000 7ff379400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 0x7ff37406fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.145+0000 7ff379400700 1 -- 172.21.15.86:0/1095216883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680095f0 con 0x7ff374106ad0 2024-03-22T19:13:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.146+0000 7ff379400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 0x7ff3741a6df0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009460 tx=0x7ff36c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.146+0000 7ff371600700 1 -- 172.21.15.86:0/1095216883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c004db0 con 0x7ff374106ad0 2024-03-22T19:13:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.146+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3740ff6c0 con 0x7ff374106ad0 2024-03-22T19:13:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.146+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3740ffbd0 con 0x7ff374106ad0 2024-03-22T19:13:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.147+0000 7ff371600700 1 -- 172.21.15.86:0/1095216883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36c01a070 con 0x7ff374106ad0 2024-03-22T19:13:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.147+0000 7ff371600700 1 -- 172.21.15.86:0/1095216883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c020690 con 0x7ff374106ad0 2024-03-22T19:13:43.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.158+0000 7ff38ef0db80 1 Processor -- start 2024-03-22T19:13:43.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.159+0000 7ff371600700 1 -- 172.21.15.86:0/1095216883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff36c00d8b0 con 0x7ff374106ad0 2024-03-22T19:13:43.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.159+0000 7ff38ef0db80 1 -- start start 2024-03-22T19:13:43.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.160+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.160+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 0x5615759ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759ec420 0x5615759ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561575943990 con 0x5615759eb580 2024-03-22T19:13:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561575941350 con 0x5615759eac20 2024-03-22T19:13:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56157593cc40 con 0x5615759ec420 2024-03-22T19:13:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff378a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 0x5615759ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff378a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 0x5615759ebe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38868/0 (socket says 172.21.15.86:38868) 2024-03-22T19:13:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.161+0000 7ff378a00700 1 -- 172.21.15.86:0/3407743804 learned_addr learned my addr 172.21.15.86:0/3407743804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.162+0000 7ff378a00700 1 -- 172.21.15.86:0/3407743804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759ec420 msgr2=0x5615759ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.162+0000 7ff378a00700 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759ec420 0x5615759ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.162+0000 7ff378a00700 1 -- 172.21.15.86:0/3407743804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 msgr2=0x5615759eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.162+0000 7ff378a00700 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.162+0000 7ff378a00700 1 -- 172.21.15.86:0/3407743804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615757be3b0 con 0x5615759eb580 2024-03-22T19:13:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.164+0000 7ff378a00700 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 0x5615759ebe40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff368015000 tx=0x7ff368009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b15f989f3405d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.165+0000 7ff365c00700 1 -- 172.21.15.86:0/3407743804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800b9d0 con 0x5615759eb580 2024-03-22T19:13:43.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.165+0000 7ff371600700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff35c0a0b10 0x7ff35c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.165+0000 7ff365c00700 1 -- 172.21.15.86:0/3407743804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36801e040 con 0x5615759eb580 2024-03-22T19:13:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.166+0000 7ff371600700 1 -- 172.21.15.86:0/1095216883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff36c01c070 con 0x7ff374106ad0 2024-03-22T19:13:43.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.168+0000 7ff365c00700 1 -- 172.21.15.86:0/3407743804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800b9d0 con 0x5615759eb580 2024-03-22T19:13:43.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.167+0000 7ff373e00700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff35c0a0b10 0x7ff35c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.168+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3407743804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 msgr2=0x5615759ebe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.168+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 0x5615759ebe40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff368015000 tx=0x7ff368009ee0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.169+0000 7ff373e00700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff35c0a0b10 0x7ff35c0a2fd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff360005fd0 tx=0x7ff360005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.169+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3407743804 shutdown_connections 2024-03-22T19:13:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.169+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759ec420 0x5615759ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.169+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.169+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3407743804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759eb580 0x5615759ebe40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.169+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3407743804 >> 172.21.15.86:0/3407743804 conn(0x561575960220 msgr2=0x5615759e5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.170+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3407743804 shutdown_connections 2024-03-22T19:13:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.170+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3407743804 wait complete. 2024-03-22T19:13:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.171+0000 7ff38ef0db80 1 -- start start 2024-03-22T19:13:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.171+0000 7ff38ef0db80 1 Processor -- start 2024-03-22T19:13:43.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.172+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615758d8d80 con 0x5615759ec420 2024-03-22T19:13:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615758d8a20 con 0x5615759eac20 2024-03-22T19:13:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615757c06a0 con 0x5615759eb580 2024-03-22T19:13:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff379400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff379400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759fbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38872/0 (socket says 172.21.15.86:38872) 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff379400700 1 -- 172.21.15.86:0/2007264958 learned_addr learned my addr 172.21.15.86:0/2007264958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.173+0000 7ff378a00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff38ef0db80 10 client.0 _ll_register_callbacks cb 0x5615757443a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff373e00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff378a00700 1 -- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 msgr2=0x5615759f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff378a00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff378a00700 1 -- 172.21.15.86:0/2007264958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 msgr2=0x5615759fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff378a00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff378a00700 1 -- 172.21.15.86:0/2007264958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff374109a90 con 0x5615759eac20 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff379400700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.174+0000 7ff378a00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f4800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff368009990 tx=0x7ff368009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c2072275bd6ed6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.175+0000 7ff35a000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.175+0000 7ff35a000700 1 -- 172.21.15.86:0/2007264958 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368003ca0 con 0x5615759eac20 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.175+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615759ddad0 con 0x5615759eac20 2024-03-22T19:13:43.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.175+0000 7ff35a000700 1 -- 172.21.15.86:0/2007264958 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff36800be80 con 0x5615759eac20 2024-03-22T19:13:43.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.176+0000 7ff35a000700 1 -- 172.21.15.86:0/2007264958 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff368003ca0 con 0x5615759eac20 2024-03-22T19:13:43.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.177+0000 7ff38ef0db80 10 client.26015 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:43.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.177+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56157576ab60 con 0x5615759eac20 2024-03-22T19:13:43.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.179+0000 7ff35a000700 1 -- 172.21.15.86:0/2007264958 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615759ddad0 con 0x5615759eac20 2024-03-22T19:13:43.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff35a000700 1 -- 172.21.15.86:0/2007264958 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff3680275c0 con 0x5615759eac20 2024-03-22T19:13:43.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff38ef0db80 10 client.26015 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:43.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff38ef0db80 10 client.26015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:13:43.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff38ef0db80 20 client.26015 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:43.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a00050 0x561575a02470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff38ef0db80 4 client.26015 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.180+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561575935700 con 0x561575a00050 2024-03-22T19:13:43.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.182+0000 7ff373e00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a00050 0x561575a02470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.188+0000 7ff35a000700 10 client.26015 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.188+0000 7ff373e00700 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a00050 0x561575a02470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da3552e75cd9ddbc server_cookie=6742154faa0389a4 in_seq=0 out_seq=0 2024-03-22T19:13:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.189+0000 7ff35a000700 1 -- 172.21.15.86:0/2007264958 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561575935700 con 0x561575a00050 2024-03-22T19:13:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.189+0000 7ff35a000700 10 client.26015 handle_command_reply: tid=0 2024-03-22T19:13:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.189+0000 7ff38ef0db80 1 client.26015 shutdown 2024-03-22T19:13:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 msgr2=0x5615759f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f4800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff368009990 tx=0x7ff368009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 shutdown_connections 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a00050 0x561575a02470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff35a000700 0 client.26015 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f4800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2007264958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.201+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 >> 172.21.15.86:0/2007264958 conn(0x561575960220 msgr2=0x5615759e51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.202+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 shutdown_connections 2024-03-22T19:13:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.202+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2007264958 wait complete. 2024-03-22T19:13:43.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.203+0000 7ff38ef0db80 20 client.26015 trim_cache size 0 max 16384 2024-03-22T19:13:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.237+0000 7ff38ef0db80 1 Processor -- start 2024-03-22T19:13:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.237+0000 7ff38ef0db80 1 -- start start 2024-03-22T19:13:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.238+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.238+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.238+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615759e0210 con 0x5615759ec420 2024-03-22T19:13:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615759fcf00 con 0x5615759eac20 2024-03-22T19:13:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff373e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff378a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615759e6d40 con 0x5615759eb580 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff378a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50500/0 (socket says 172.21.15.86:50500) 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.239+0000 7ff378a00700 1 -- 172.21.15.86:0/3837033385 learned_addr learned my addr 172.21.15.86:0/3837033385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.240+0000 7ff378a00700 1 -- 172.21.15.86:0/3837033385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 msgr2=0x5615759ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.240+0000 7ff378a00700 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.241+0000 7ff378a00700 1 -- 172.21.15.86:0/3837033385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 msgr2=0x5615759e51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.241+0000 7ff379400700 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.241+0000 7ff378a00700 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759e51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.241+0000 7ff378a00700 1 -- 172.21.15.86:0/3837033385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561575902ea0 con 0x5615759eac20 2024-03-22T19:13:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.242+0000 7ff373e00700 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.242+0000 7ff379400700 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759e51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.244+0000 7ff378a00700 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f9d30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff368005950 tx=0x7ff36800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fa049578a14af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.244+0000 7ff366600700 1 -- 172.21.15.86:0/3837033385 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368016e20 con 0x5615759eac20 2024-03-22T19:13:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.244+0000 7ff366600700 1 -- 172.21.15.86:0/3837033385 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3680050e0 con 0x5615759eac20 2024-03-22T19:13:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.244+0000 7ff366600700 1 -- 172.21.15.86:0/3837033385 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368016e20 con 0x5615759eac20 2024-03-22T19:13:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3837033385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 msgr2=0x5615759f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f9d30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff368005950 tx=0x7ff36800b560 comp rx=0 tx=0).stop 2024-03-22T19:13:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3837033385 shutdown_connections 2024-03-22T19:13:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759e51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f9d30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/3837033385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.249+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3837033385 >> 172.21.15.86:0/3837033385 conn(0x561575960220 msgr2=0x5615759e5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.250+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3837033385 shutdown_connections 2024-03-22T19:13:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.250+0000 7ff38ef0db80 1 -- 172.21.15.86:0/3837033385 wait complete. 2024-03-22T19:13:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.251+0000 7ff38ef0db80 1 -- start start 2024-03-22T19:13:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.251+0000 7ff38ef0db80 1 Processor -- start 2024-03-22T19:13:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.255+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.256+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.256+0000 7ff38ef0db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.256+0000 7ff378a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.257+0000 7ff379400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.258+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615759f9580 con 0x5615759ec420 2024-03-22T19:13:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.258+0000 7ff38ef0db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615759f96f0 con 0x5615759eac20 2024-03-22T19:13:43.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.258+0000 7ff378a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50508/0 (socket says 172.21.15.86:50508) 2024-03-22T19:13:43.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.258+0000 7ff378a00700 1 -- 172.21.15.86:0/2832917115 learned_addr learned my addr 172.21.15.86:0/2832917115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.258+0000 7ff373e00700 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.259+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615759f98b0 con 0x5615759eb580 2024-03-22T19:13:43.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.259+0000 7ff38ef0db80 10 client.0 _ll_register_callbacks cb 0x7ff368005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:43.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.261+0000 7ff378a00700 1 -- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 msgr2=0x5615759f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.261+0000 7ff378a00700 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.261+0000 7ff378a00700 1 -- 172.21.15.86:0/2832917115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 msgr2=0x5615759f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.261+0000 7ff378a00700 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.261+0000 7ff378a00700 1 -- 172.21.15.86:0/2832917115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56157576ab60 con 0x5615759eac20 2024-03-22T19:13:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.262+0000 7ff378a00700 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f7be0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff368005ae0 tx=0x7ff368005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e20e6f28bb388fd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.262+0000 7ff35a000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.263+0000 7ff35a000700 1 -- 172.21.15.86:0/2832917115 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3680165b0 con 0x5615759eac20 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.264+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff36801e040 con 0x5615759eac20 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.264+0000 7ff35a000700 1 -- 172.21.15.86:0/2832917115 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff368000e40 con 0x5615759eac20 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.264+0000 7ff35a000700 1 -- 172.21.15.86:0/2832917115 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3680165b0 con 0x5615759eac20 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.264+0000 7ff38ef0db80 10 client.26021 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.264+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5615757c2cd0 con 0x5615759eac20 2024-03-22T19:13:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.266+0000 7ff35a000700 1 -- 172.21.15.86:0/2832917115 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36801e040 con 0x5615759eac20 2024-03-22T19:13:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.266+0000 7ff35a000700 1 -- 172.21.15.86:0/2832917115 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff36800fc20 con 0x5615759eac20 2024-03-22T19:13:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.266+0000 7ff38ef0db80 10 client.26021 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.267+0000 7ff38ef0db80 10 client.26021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:13:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.267+0000 7ff38ef0db80 20 client.26021 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.267+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a526f0 0x561575a54b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.267+0000 7ff38ef0db80 4 client.26021 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.267+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56157578bcf0 con 0x561575a526f0 2024-03-22T19:13:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.268+0000 7ff373e00700 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a526f0 0x561575a54b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.276+0000 7ff373e00700 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a526f0 0x561575a54b10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95835f4971be8ca7 server_cookie=f8fae662dc6f1325 in_seq=0 out_seq=0 2024-03-22T19:13:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.277+0000 7ff35a000700 10 client.26021 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.277+0000 7ff35a000700 1 -- 172.21.15.86:0/2832917115 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56157578bcf0 con 0x561575a526f0 2024-03-22T19:13:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.277+0000 7ff35a000700 10 client.26021 handle_command_reply: tid=0 2024-03-22T19:13:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.277+0000 7ff38ef0db80 1 client.26021 shutdown 2024-03-22T19:13:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 msgr2=0x5615759f7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f7be0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff368005ae0 tx=0x7ff368005cc0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 shutdown_connections 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x561575a526f0 0x561575a54b10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615759ec420 0x5615759f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615759eac20 0x5615759f7be0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff35a000700 0 client.26021 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 --2- 172.21.15.86:0/2832917115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615759eb580 0x5615759f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.278+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 >> 172.21.15.86:0/2832917115 conn(0x561575960220 msgr2=0x5615759eef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.279+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 shutdown_connections 2024-03-22T19:13:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.279+0000 7ff38ef0db80 1 -- 172.21.15.86:0/2832917115 wait complete. 2024-03-22T19:13:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.280+0000 7ff38ef0db80 20 client.26021 trim_cache size 0 max 16384 2024-03-22T19:13:43.281 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff35c0a0b10 msgr2=0x7ff35c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff35c0a0b10 0x7ff35c0a2fd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff360005fd0 tx=0x7ff360005f60 comp rx=0 tx=0).stop 2024-03-22T19:13:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 msgr2=0x7ff3741a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 0x7ff3741a6df0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009460 tx=0x7ff36c00d3b0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 shutdown_connections 2024-03-22T19:13:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff35c0a0b10 0x7ff35c0a2fd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106ad0 0x7ff3741a6df0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff374101870 0x7ff37406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 --2- 172.21.15.86:0/1095216883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3741041a0 0x7ff37406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.282+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 >> 172.21.15.86:0/1095216883 conn(0x7ff3740fb1a0 msgr2=0x7ff374102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.284+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 shutdown_connections 2024-03-22T19:13:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.285+0000 7ff37b400700 1 -- 172.21.15.86:0/1095216883 wait complete. 2024-03-22T19:13:43.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:13:43.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:43 smithi131 ceph-mon[78559]: pgmap v368: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 182 MiB/s wr, 20.70k op/s 2024-03-22T19:13:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:43 smithi086 ceph-mon[38032]: pgmap v368: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 182 MiB/s wr, 20.70k op/s 2024-03-22T19:13:43.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.762+0000 7f6426c00700 1 -- 172.21.15.86:0/1768979624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6420074d90 msgr2=0x7f64200731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.762+0000 7f6426c00700 1 --2- 172.21.15.86:0/1768979624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6420074d90 0x7f64200731e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6414009990 tx=0x7f6414009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.763+0000 7f6426c00700 1 -- 172.21.15.86:0/1768979624 shutdown_connections 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.764+0000 7f6426c00700 1 --2- 172.21.15.86:0/1768979624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6420108a50 0x7f642010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.764+0000 7f6426c00700 1 --2- 172.21.15.86:0/1768979624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6420074d90 0x7f64200731e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.764+0000 7f6426c00700 1 --2- 172.21.15.86:0/1768979624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64200737b0 0x7f6420073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.764+0000 7f6426c00700 1 -- 172.21.15.86:0/1768979624 >> 172.21.15.86:0/1768979624 conn(0x7f64200fc0f0 msgr2=0x7f64200fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.764+0000 7f6426c00700 1 -- 172.21.15.86:0/1768979624 shutdown_connections 2024-03-22T19:13:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.764+0000 7f6426c00700 1 -- 172.21.15.86:0/1768979624 wait complete. 2024-03-22T19:13:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.765+0000 7f6426c00700 1 Processor -- start 2024-03-22T19:13:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 -- start start 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 0x7f6420198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6420074d90 0x7f6420199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6420108a50 0x7f64201a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64201a0820 con 0x7f6420074d90 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64201a0990 con 0x7f64200737b0 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.766+0000 7f6426c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64201a0b50 con 0x7f6420108a50 2024-03-22T19:13:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.767+0000 7f6424e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6420074d90 0x7f6420199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.767+0000 7f6425800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 0x7f6420198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.767+0000 7f6425800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 0x7f6420198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50512/0 (socket says 172.21.15.86:50512) 2024-03-22T19:13:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.767+0000 7f6425800700 1 -- 172.21.15.86:0/987265837 learned_addr learned my addr 172.21.15.86:0/987265837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.768+0000 7f6425800700 1 -- 172.21.15.86:0/987265837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6420108a50 msgr2=0x7f64201a0250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:13:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.768+0000 7f6425800700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6420108a50 0x7f64201a0250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.768+0000 7f6425800700 1 -- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6420074d90 msgr2=0x7f6420199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.768+0000 7f6425800700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6420074d90 0x7f6420199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.768+0000 7f6425800700 1 -- 172.21.15.86:0/987265837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140095f0 con 0x7f64200737b0 2024-03-22T19:13:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.768+0000 7f6425800700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 0x7f6420198d00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6414000c00 tx=0x7f6414005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.769+0000 7f641e000700 1 -- 172.21.15.86:0/987265837 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641401d070 con 0x7f64200737b0 2024-03-22T19:13:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.769+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64201a0dd0 con 0x7f64200737b0 2024-03-22T19:13:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.769+0000 7f641e000700 1 -- 172.21.15.86:0/987265837 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641400fb80 con 0x7f64200737b0 2024-03-22T19:13:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.769+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64201a11a0 con 0x7f64200737b0 2024-03-22T19:13:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.770+0000 7f641e000700 1 -- 172.21.15.86:0/987265837 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641401d070 con 0x7f64200737b0 2024-03-22T19:13:43.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.775+0000 7f643bdbab80 1 Processor -- start 2024-03-22T19:13:43.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 -- start start 2024-03-22T19:13:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e504220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e5054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e50c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e45c990 con 0x56412e5059e0 2024-03-22T19:13:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e45a350 con 0x56412e505020 2024-03-22T19:13:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.776+0000 7f643bdbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e455c40 con 0x56412e503de0 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6426200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e504220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6426200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e504220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59922/0 (socket says 172.21.15.86:59922) 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6426200700 1 -- 172.21.15.86:0/843793239 learned_addr learned my addr 172.21.15.86:0/843793239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6424e00700 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e50c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6426200700 1 -- 172.21.15.86:0/843793239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 msgr2=0x56412e5054a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6426200700 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e5054a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.777+0000 7f6426200700 1 -- 172.21.15.86:0/843793239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 msgr2=0x56412e50c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.778+0000 7f6426200700 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e50c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.778+0000 7f6426200700 1 -- 172.21.15.86:0/843793239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56412e2d73b0 con 0x56412e503de0 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.778+0000 7f6424e00700 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e50c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.779+0000 7f6426200700 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e504220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6418009be0 tx=0x7f6418009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca3f94acff821c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.779+0000 7f6412a00700 1 -- 172.21.15.86:0/843793239 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418004bb0 con 0x56412e503de0 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.779+0000 7f6412a00700 1 -- 172.21.15.86:0/843793239 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641800f040 con 0x56412e503de0 2024-03-22T19:13:43.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.779+0000 7f6412a00700 1 -- 172.21.15.86:0/843793239 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418013650 con 0x56412e503de0 2024-03-22T19:13:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.779+0000 7f641e000700 1 -- 172.21.15.86:0/987265837 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f641400fcf0 con 0x7f64200737b0 2024-03-22T19:13:43.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.782+0000 7f641e000700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64080a0b70 0x7f64080a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.782+0000 7f641e000700 1 -- 172.21.15.86:0/987265837 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f64140ccf30 con 0x7f64200737b0 2024-03-22T19:13:43.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.783+0000 7f643bdbab80 1 -- 172.21.15.86:0/843793239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 msgr2=0x56412e504220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.783+0000 7f643bdbab80 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e504220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6418009be0 tx=0x7f6418009ef0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.784+0000 7f643bdbab80 1 -- 172.21.15.86:0/843793239 shutdown_connections 2024-03-22T19:13:43.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.784+0000 7f643bdbab80 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e50c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.784+0000 7f643bdbab80 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e504220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.784+0000 7f643bdbab80 1 --2- 172.21.15.86:0/843793239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e5054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.784+0000 7f643bdbab80 1 -- 172.21.15.86:0/843793239 >> 172.21.15.86:0/843793239 conn(0x56412e479220 msgr2=0x56412e4fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.784+0000 7f6425800700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64080a0b70 0x7f64080a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.785+0000 7f643bdbab80 1 -- 172.21.15.86:0/843793239 shutdown_connections 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.785+0000 7f6425800700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64080a0b70 0x7f64080a3030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64140a2590 tx=0x7f6414009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.785+0000 7f643bdbab80 1 -- 172.21.15.86:0/843793239 wait complete. 2024-03-22T19:13:43.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.786+0000 7f643bdbab80 1 -- start start 2024-03-22T19:13:43.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.786+0000 7f643bdbab80 1 Processor -- start 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.786+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e50dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e50e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e515270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f643bdbab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e3f1d80 con 0x56412e5059e0 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f643bdbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e3f1a20 con 0x56412e503de0 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f643bdbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e2d96a0 con 0x56412e505020 2024-03-22T19:13:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f643bdbab80 10 client.0 _ll_register_callbacks cb 0x56412e25d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:43.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f6426200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e50dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f6426200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e50dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50524/0 (socket says 172.21.15.86:50524) 2024-03-22T19:13:43.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f6426200700 1 -- 172.21.15.86:0/3670854935 learned_addr learned my addr 172.21.15.86:0/3670854935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f6426200700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e515270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.787+0000 7f6424e00700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e50e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6426200700 1 -- 172.21.15.86:0/3670854935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 msgr2=0x56412e50e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6426200700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e50e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6426200700 1 -- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 msgr2=0x56412e515270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6426200700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e515270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6426200700 1 -- 172.21.15.86:0/3670854935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64180098c0 con 0x56412e503de0 2024-03-22T19:13:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6424e00700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e50e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6426200700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e50dcf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6418005bd0 tx=0x7f6418005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e707cbeec59fbd8e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.788+0000 7f6407400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f6407400700 1 -- 172.21.15.86:0/3670854935 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418021880 con 0x56412e503de0 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f6407400700 1 -- 172.21.15.86:0/3670854935 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f64180219e0 con 0x56412e503de0 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56412e4f6c30 con 0x56412e503de0 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f6407400700 1 -- 172.21.15.86:0/3670854935 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6418021ba0 con 0x56412e503de0 2024-03-22T19:13:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f643bdbab80 10 client.26030 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56412e283b60 con 0x56412e503de0 2024-03-22T19:13:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.789+0000 7f6407400700 1 -- 172.21.15.86:0/3670854935 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6418031020 con 0x56412e503de0 2024-03-22T19:13:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f6407400700 1 -- 172.21.15.86:0/3670854935 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f641801f460 con 0x56412e503de0 2024-03-22T19:13:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f643bdbab80 10 client.26030 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f643bdbab80 10 client.26030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:13:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f643bdbab80 20 client.26030 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e519930 0x56412e51bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f643bdbab80 4 client.26030 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.790+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56412e44e700 con 0x56412e519930 2024-03-22T19:13:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.791+0000 7f6424e00700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e519930 0x56412e51bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.792+0000 7f6407400700 10 client.26030 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:43.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.792+0000 7f6424e00700 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e519930 0x56412e51bd50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6269ce1a9746b7c server_cookie=525776408ecc8b34 in_seq=0 out_seq=0 2024-03-22T19:13:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.795+0000 7f6407400700 1 -- 172.21.15.86:0/3670854935 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56412e44e700 con 0x56412e519930 2024-03-22T19:13:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.795+0000 7f6407400700 10 client.26030 handle_command_reply: tid=0 2024-03-22T19:13:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.795+0000 7f643bdbab80 1 client.26030 shutdown 2024-03-22T19:13:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 msgr2=0x56412e50dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e50dcf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6418005bd0 tx=0x7f6418005b40 comp rx=0 tx=0).stop 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 shutdown_connections 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e519930 0x56412e51bd50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e515270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e50dcf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3670854935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e50e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 >> 172.21.15.86:0/3670854935 conn(0x56412e479220 msgr2=0x56412e4fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f6407400700 0 client.26030 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 shutdown_connections 2024-03-22T19:13:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.796+0000 7f643bdbab80 1 -- 172.21.15.86:0/3670854935 wait complete. 2024-03-22T19:13:43.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.797+0000 7f643bdbab80 20 client.26030 trim_cache size 0 max 16384 2024-03-22T19:13:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.824+0000 7f643bdbab80 1 Processor -- start 2024-03-22T19:13:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.824+0000 7f643bdbab80 1 -- start start 2024-03-22T19:13:43.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:43 smithi167 ceph-mon[47728]: pgmap v368: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 182 MiB/s wr, 20.70k op/s 2024-03-22T19:13:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e5206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e513220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e4fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f643bdbab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e505e00 con 0x56412e5059e0 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f643bdbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e505820 con 0x56412e503de0 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f643bdbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e50eee0 con 0x56412e505020 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.827+0000 7f6426200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e5206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e5206b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50536/0 (socket says 172.21.15.86:50536) 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 -- 172.21.15.86:0/3441697966 learned_addr learned my addr 172.21.15.86:0/3441697966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6424e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e513220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 -- 172.21.15.86:0/3441697966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 msgr2=0x56412e513220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e513220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 -- 172.21.15.86:0/3441697966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 msgr2=0x56412e4fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e4fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.828+0000 7f6426200700 1 -- 172.21.15.86:0/3441697966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56412e41bea0 con 0x56412e503de0 2024-03-22T19:13:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.829+0000 7f6426200700 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e5206b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64180097e0 tx=0x7f64180091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602931c8e06638e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.830+0000 7f6413400700 1 -- 172.21.15.86:0/3441697966 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418004b90 con 0x56412e503de0 2024-03-22T19:13:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.830+0000 7f6413400700 1 -- 172.21.15.86:0/3441697966 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6418031070 con 0x56412e503de0 2024-03-22T19:13:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.830+0000 7f6413400700 1 -- 172.21.15.86:0/3441697966 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418021b40 con 0x56412e503de0 2024-03-22T19:13:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 -- 172.21.15.86:0/3441697966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 msgr2=0x56412e5206b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e5206b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64180097e0 tx=0x7f64180091b0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 -- 172.21.15.86:0/3441697966 shutdown_connections 2024-03-22T19:13:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e4fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e503de0 0x56412e5206b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 --2- 172.21.15.86:0/3441697966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e505020 0x56412e513220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.832+0000 7f643bdbab80 1 -- 172.21.15.86:0/3441697966 >> 172.21.15.86:0/3441697966 conn(0x56412e479220 msgr2=0x56412e50d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.833+0000 7f643bdbab80 1 -- 172.21.15.86:0/3441697966 shutdown_connections 2024-03-22T19:13:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.833+0000 7f643bdbab80 1 -- 172.21.15.86:0/3441697966 wait complete. 2024-03-22T19:13:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.833+0000 7f643bdbab80 1 -- start start 2024-03-22T19:13:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.833+0000 7f643bdbab80 1 Processor -- start 2024-03-22T19:13:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.835+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e56dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.835+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e570170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f643bdbab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e572760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f643bdbab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e572ec0 con 0x56412e5059e0 2024-03-22T19:13:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f643bdbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e573030 con 0x56412e505020 2024-03-22T19:13:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f643bdbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412e5731f0 con 0x56412e503de0 2024-03-22T19:13:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f643bdbab80 10 client.0 _ll_register_callbacks cb 0x7f6418005330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f6424e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e570170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f6424e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e570170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50540/0 (socket says 172.21.15.86:50540) 2024-03-22T19:13:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f6424e00700 1 -- 172.21.15.86:0/2498255094 learned_addr learned my addr 172.21.15.86:0/2498255094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.836+0000 7f6426200700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e56dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.838+0000 7f6424e00700 1 -- 172.21.15.86:0/2498255094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 msgr2=0x56412e56dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.838+0000 7f6424e00700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e56dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.838+0000 7f6424e00700 1 -- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 msgr2=0x56412e572760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.838+0000 7f6424e00700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e572760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.838+0000 7f6424e00700 1 -- 172.21.15.86:0/2498255094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56412e283b60 con 0x56412e505020 2024-03-22T19:13:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.838+0000 7f6426200700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e56dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.839+0000 7f6424e00700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e570170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56412e51c630 tx=0x7f640c003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda7473bebd11b12 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.839+0000 7f6407400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.840+0000 7f6407400700 1 -- 172.21.15.86:0/2498255094 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c017040 con 0x56412e505020 2024-03-22T19:13:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.840+0000 7f6407400700 1 -- 172.21.15.86:0/2498255094 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f640c0097c0 con 0x56412e505020 2024-03-22T19:13:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.840+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f641800f040 con 0x56412e505020 2024-03-22T19:13:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.841+0000 7f6407400700 1 -- 172.21.15.86:0/2498255094 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f640c009980 con 0x56412e505020 2024-03-22T19:13:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.842+0000 7f643bdbab80 10 client.26036 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.842+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56412e2dbcd0 con 0x56412e505020 2024-03-22T19:13:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.842+0000 7f6407400700 1 -- 172.21.15.86:0/2498255094 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641800f040 con 0x56412e505020 2024-03-22T19:13:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.842+0000 7f6407400700 1 -- 172.21.15.86:0/2498255094 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f640c02d070 con 0x56412e505020 2024-03-22T19:13:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.843+0000 7f643bdbab80 10 client.26036 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.843+0000 7f643bdbab80 10 client.26036 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:13:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.843+0000 7f643bdbab80 20 client.26036 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.843+0000 7f643bdbab80 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e574c20 0x56412e577040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.843+0000 7f643bdbab80 4 client.26036 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.843+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56412e2a4cf0 con 0x56412e574c20 2024-03-22T19:13:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.846+0000 7f6426200700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e574c20 0x56412e577040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.849+0000 7f6407400700 10 client.26036 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.849+0000 7f6426200700 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e574c20 0x56412e577040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=348833a8e1d236e server_cookie=87592d48c1f586d5 in_seq=0 out_seq=0 2024-03-22T19:13:43.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f6407400700 1 -- 172.21.15.86:0/2498255094 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56412e2a4cf0 con 0x56412e574c20 2024-03-22T19:13:43.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f6407400700 10 client.26036 handle_command_reply: tid=0 2024-03-22T19:13:43.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 client.26036 shutdown 2024-03-22T19:13:43.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 msgr2=0x56412e570170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e570170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56412e51c630 tx=0x7f640c003ed0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 shutdown_connections 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x56412e574c20 0x56412e577040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412e5059e0 0x56412e572760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56412e503de0 0x56412e56dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 --2- 172.21.15.86:0/2498255094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56412e505020 0x56412e570170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 >> 172.21.15.86:0/2498255094 conn(0x56412e479220 msgr2=0x56412e4fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.850+0000 7f6407400700 0 client.26036 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:13:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.851+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 shutdown_connections 2024-03-22T19:13:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.852+0000 7f643bdbab80 1 -- 172.21.15.86:0/2498255094 wait complete. 2024-03-22T19:13:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.853+0000 7f643bdbab80 20 client.26036 trim_cache size 0 max 16384 2024-03-22T19:13:43.854 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.853+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64080a0b70 msgr2=0x7f64080a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.853+0000 7f6426c00700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64080a0b70 0x7f64080a3030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64140a2590 tx=0x7f6414009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.853+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 msgr2=0x7f6420198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 0x7f6420198d00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6414000c00 tx=0x7f6414005ef0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 shutdown_connections 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f64080a0b70 0x7f64080a3030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6420108a50 0x7f64201a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64200737b0 0x7f6420198d00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 --2- 172.21.15.86:0/987265837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6420074d90 0x7f6420199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 >> 172.21.15.86:0/987265837 conn(0x7f64200fc0f0 msgr2=0x7f64201049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 shutdown_connections 2024-03-22T19:13:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:43.854+0000 7f6426c00700 1 -- 172.21.15.86:0/987265837 wait complete. 2024-03-22T19:13:43.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:13:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.266+0000 7f63dfc00700 1 -- 172.21.15.86:0/3878142313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 msgr2=0x7f63d8072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.266+0000 7f63dfc00700 1 --2- 172.21.15.86:0/3878142313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d8072600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63d001c0d0 tx=0x7f63d001c3e0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.268+0000 7f63dfc00700 1 -- 172.21.15.86:0/3878142313 shutdown_connections 2024-03-22T19:13:44.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.268+0000 7f63dfc00700 1 --2- 172.21.15.86:0/3878142313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8108ee0 0x7f63d810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.268+0000 7f63dfc00700 1 --2- 172.21.15.86:0/3878142313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d8072600 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.268+0000 7f63dfc00700 1 --2- 172.21.15.86:0/3878142313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108520 0x7f63d81089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.268+0000 7f63dfc00700 1 -- 172.21.15.86:0/3878142313 >> 172.21.15.86:0/3878142313 conn(0x7f63d806dd20 msgr2=0x7f63d8070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.268+0000 7f63dfc00700 1 -- 172.21.15.86:0/3878142313 shutdown_connections 2024-03-22T19:13:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.269+0000 7f63dfc00700 1 -- 172.21.15.86:0/3878142313 wait complete. 2024-03-22T19:13:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.270+0000 7f63dfc00700 1 Processor -- start 2024-03-22T19:13:44.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.271+0000 7f63dfc00700 1 -- start start 2024-03-22T19:13:44.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.273+0000 7f63dfc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.273+0000 7f63dfc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8108520 0x7f63d819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.274+0000 7f63dfc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108ee0 0x7f63d81a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.274+0000 7f63dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.275+0000 7f63dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50554/0 (socket says 172.21.15.86:50554) 2024-03-22T19:13:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.275+0000 7f63dfc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81a4f60 con 0x7f63d8108520 2024-03-22T19:13:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.275+0000 7f63dfc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81a50a0 con 0x7f63d80721e0 2024-03-22T19:13:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.275+0000 7f63dfc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81a5230 con 0x7f63d8108ee0 2024-03-22T19:13:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.276+0000 7f63dd200700 1 -- 172.21.15.86:0/760385067 learned_addr learned my addr 172.21.15.86:0/760385067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.276+0000 7f63ddc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108ee0 0x7f63d81a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.276+0000 7f63d7e00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8108520 0x7f63d819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.277+0000 7f63ddc00700 1 -- 172.21.15.86:0/760385067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 msgr2=0x7f63d819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.277+0000 7f63ddc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.277+0000 7f63ddc00700 1 -- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8108520 msgr2=0x7f63d819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.277+0000 7f63ddc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8108520 0x7f63d819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.277+0000 7f63ddc00700 1 -- 172.21.15.86:0/760385067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d001ad10 con 0x7f63d8108ee0 2024-03-22T19:13:44.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.277+0000 7f63dd200700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.283+0000 7f63ddc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108ee0 0x7f63d81a4990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c970 tx=0x7f63c800cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.283+0000 7f63d5600700 1 -- 172.21.15.86:0/760385067 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007b00 con 0x7f63d8108ee0 2024-03-22T19:13:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.284+0000 7f63d5600700 1 -- 172.21.15.86:0/760385067 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c800ce50 con 0x7f63d8108ee0 2024-03-22T19:13:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.284+0000 7f63d5600700 1 -- 172.21.15.86:0/760385067 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8018660 con 0x7f63d8108ee0 2024-03-22T19:13:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.284+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d81a5510 con 0x7f63d8108ee0 2024-03-22T19:13:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.284+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d81a59d0 con 0x7f63d8108ee0 2024-03-22T19:13:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.295+0000 7f63f3710b80 1 Processor -- start 2024-03-22T19:13:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.296+0000 7f63d5600700 1 -- 172.21.15.86:0/760385067 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f63c80213f0 con 0x7f63d8108ee0 2024-03-22T19:13:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.300+0000 7f63d5600700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f63bc0a0b10 0x7f63bc0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.300+0000 7f63f3710b80 1 -- start start 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215071c20 0x557215072040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215072580 0x557215072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63f3710b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214fca990 con 0x557215071c20 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63f3710b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214fc8350 con 0x557215073420 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63f3710b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214fc3c40 con 0x557215072580 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63d7e00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f63bc0a0b10 0x7f63bc0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.302+0000 7f63d7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215072580 0x557215072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.302+0000 7f63d7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215072580 0x557215072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:59986/0 (socket says 172.21.15.86:59986) 2024-03-22T19:13:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.302+0000 7f63d7e00700 1 -- 172.21.15.86:0/2118032498 learned_addr learned my addr 172.21.15.86:0/2118032498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.303+0000 7f63ddc00700 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.303+0000 7f63dd200700 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215071c20 0x557215072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.303+0000 7f63ddc00700 1 -- 172.21.15.86:0/2118032498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215072580 msgr2=0x557215072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.303+0000 7f63ddc00700 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215072580 0x557215072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.303+0000 7f63ddc00700 1 -- 172.21.15.86:0/2118032498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215071c20 msgr2=0x557215072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.304+0000 7f63d7e00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f63bc0a0b10 0x7f63bc0a2fd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f63c4013a40 tx=0x7f63c400b850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.304+0000 7f63ddc00700 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215071c20 0x557215072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.301+0000 7f63d5600700 1 -- 172.21.15.86:0/760385067 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f63c80cbd70 con 0x7f63d8108ee0 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.304+0000 7f63ddc00700 1 -- 172.21.15.86:0/2118032498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557214e453b0 con 0x557215073420 2024-03-22T19:13:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.304+0000 7f63ddc00700 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150738a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f63c80c9950 tx=0x7f63c80990d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521488b0bbe370f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.306+0000 7f63ce000700 1 -- 172.21.15.86:0/2118032498 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8099b90 con 0x557215073420 2024-03-22T19:13:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.306+0000 7f63ce000700 1 -- 172.21.15.86:0/2118032498 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c8099490 con 0x557215073420 2024-03-22T19:13:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.306+0000 7f63ce000700 1 -- 172.21.15.86:0/2118032498 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8095dc0 con 0x557215073420 2024-03-22T19:13:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.306+0000 7f63dd200700 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215071c20 0x557215072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.309+0000 7f63f3710b80 1 -- 172.21.15.86:0/2118032498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 msgr2=0x5572150738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.309+0000 7f63f3710b80 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150738a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f63c80c9950 tx=0x7f63c80990d0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 -- 172.21.15.86:0/2118032498 shutdown_connections 2024-03-22T19:13:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150738a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215071c20 0x557215072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 --2- 172.21.15.86:0/2118032498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215072580 0x557215072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 -- 172.21.15.86:0/2118032498 >> 172.21.15.86:0/2118032498 conn(0x557214fe7220 msgr2=0x55721506cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 -- 172.21.15.86:0/2118032498 shutdown_connections 2024-03-22T19:13:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.314+0000 7f63f3710b80 1 -- 172.21.15.86:0/2118032498 wait complete. 2024-03-22T19:13:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.316+0000 7f63f3710b80 1 -- start start 2024-03-22T19:13:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.316+0000 7f63f3710b80 1 Processor -- start 2024-03-22T19:13:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.317+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x55721507bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.317+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x55721507c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63dd200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x55721507bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x55721507c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x55721507c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38996/0 (socket says 172.21.15.86:38996) 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63dd200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x55721507bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60000/0 (socket says 172.21.15.86:60000) 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63d7e00700 1 -- 172.21.15.86:0/3301131498 learned_addr learned my addr 172.21.15.86:0/3301131498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x557215083190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214f5fd80 con 0x557215072580 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214f5fa20 con 0x557215073420 2024-03-22T19:13:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.318+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214e476a0 con 0x557215071c20 2024-03-22T19:13:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.319+0000 7f63f3710b80 10 client.0 _ll_register_callbacks cb 0x557214dcb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.319+0000 7f63d7e00700 1 -- 172.21.15.86:0/3301131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 msgr2=0x55721507bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.319+0000 7f63d7e00700 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x55721507bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.319+0000 7f63d7e00700 1 -- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 msgr2=0x557215083190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.319+0000 7f63d7e00700 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x557215083190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.319+0000 7f63d7e00700 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d81a59d0 con 0x557215072580 2024-03-22T19:13:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.320+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557215064ad0 con 0x557215072580 2024-03-22T19:13:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.334+0000 7f63d7e00700 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x55721507c150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f63c4011a20 tx=0x7f63c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41112fac1a5e150f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.334+0000 7f63c2a00700 10 client.15963 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:13:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.335+0000 7f63c2a00700 1 -- 172.21.15.86:0/3301131498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011e90 con 0x557215072580 2024-03-22T19:13:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.335+0000 7f63c2a00700 1 -- 172.21.15.86:0/3301131498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f63c4024030 con 0x557215072580 2024-03-22T19:13:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.335+0000 7f63c2a00700 1 -- 172.21.15.86:0/3301131498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c400f9e0 con 0x557215072580 2024-03-22T19:13:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.336+0000 7f63f3710b80 10 client.15963 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.336+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x557214df1b60 con 0x557215072580 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.336+0000 7f63c2a00700 1 -- 172.21.15.86:0/3301131498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f63c400fb40 con 0x557215072580 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.336+0000 7f63f3710b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.336+0000 7f63f3710b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.337+0000 7f63f3710b80 20 client.15963 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.337+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557215087870 0x557215089c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.337+0000 7f63f3710b80 4 client.15963 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.337+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557214fbc700 con 0x557215087870 2024-03-22T19:13:44.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.337+0000 7f63dd200700 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557215087870 0x557215089c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.338+0000 7f63dd200700 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557215087870 0x557215089c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee64fa12152b45df server_cookie=5305cda977ec8b41 in_seq=0 out_seq=0 2024-03-22T19:13:44.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.338+0000 7f63c2a00700 10 client.15963 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:44.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.339+0000 7f63c2a00700 1 -- 172.21.15.86:0/3301131498 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557214fbc700 con 0x557215087870 2024-03-22T19:13:44.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.339+0000 7f63c2a00700 10 client.15963 handle_command_reply: tid=0 2024-03-22T19:13:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.340+0000 7f63f3710b80 1 client.15963 shutdown 2024-03-22T19:13:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.340+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 msgr2=0x55721507c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.340+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x55721507c150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f63c4011a20 tx=0x7f63c40047c0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 shutdown_connections 2024-03-22T19:13:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557215087870 0x557215089c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63c2a00700 0 client.15963 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x557215083190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x55721507bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 --2- 172.21.15.86:0/3301131498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x55721507c150 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 >> 172.21.15.86:0/3301131498 conn(0x557214fe7220 msgr2=0x55721506c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 shutdown_connections 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.342+0000 7f63f3710b80 1 -- 172.21.15.86:0/3301131498 wait complete. 2024-03-22T19:13:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.343+0000 7f63f3710b80 20 client.15963 trim_cache size 0 max 16384 2024-03-22T19:13:44.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 Processor -- start 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 -- start start 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x5572150810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x557215073820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x55721507a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557215084310 con 0x557215072580 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572150695e0 con 0x557215073420 2024-03-22T19:13:44.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.372+0000 7f63f3710b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721506d930 con 0x557215071c20 2024-03-22T19:13:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.373+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x55721507a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.373+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x55721507a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50566/0 (socket says 172.21.15.86:50566) 2024-03-22T19:13:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.373+0000 7f63ddc00700 1 -- 172.21.15.86:0/4293850669 learned_addr learned my addr 172.21.15.86:0/4293850669 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63dd200700 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x5572150810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63d7e00700 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x557215073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63dd200700 1 -- 172.21.15.86:0/4293850669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 msgr2=0x55721507a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63dd200700 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x55721507a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63dd200700 1 -- 172.21.15.86:0/4293850669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 msgr2=0x557215073820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63dd200700 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x557215073820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.374+0000 7f63dd200700 1 -- 172.21.15.86:0/4293850669 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557214f89ea0 con 0x557215071c20 2024-03-22T19:13:44.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.375+0000 7f63dd200700 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x5572150810a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63d001c860 tx=0x7f63d001ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d6e218e7d28864 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:44.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.376+0000 7f63cea00700 1 -- 172.21.15.86:0/4293850669 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d001f040 con 0x557215071c20 2024-03-22T19:13:44.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.376+0000 7f63cea00700 1 -- 172.21.15.86:0/4293850669 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d0031070 con 0x557215071c20 2024-03-22T19:13:44.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.376+0000 7f63cea00700 1 -- 172.21.15.86:0/4293850669 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0031c70 con 0x557215071c20 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.378+0000 7f63f3710b80 1 -- 172.21.15.86:0/4293850669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 msgr2=0x5572150810a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.378+0000 7f63f3710b80 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x5572150810a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63d001c860 tx=0x7f63d001ac30 comp rx=0 tx=0).stop 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 -- 172.21.15.86:0/4293850669 shutdown_connections 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x55721507a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557215071c20 0x5572150810a0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63d001c860 tx=0x7f63d001ac30 comp rx=0 tx=0).stop 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 --2- 172.21.15.86:0/4293850669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x557215073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 -- 172.21.15.86:0/4293850669 >> 172.21.15.86:0/4293850669 conn(0x557214fe7220 msgr2=0x5572150763e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 -- 172.21.15.86:0/4293850669 shutdown_connections 2024-03-22T19:13:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.379+0000 7f63f3710b80 1 -- 172.21.15.86:0/4293850669 wait complete. 2024-03-22T19:13:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.380+0000 7f63f3710b80 1 -- start start 2024-03-22T19:13:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.380+0000 7f63f3710b80 1 Processor -- start 2024-03-22T19:13:44.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.383+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x5572150d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.384+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.384+0000 7f63f3710b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 0x5572150d8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63f3710b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572150d92b0 con 0x557215072580 2024-03-22T19:13:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63f3710b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572150d9420 con 0x557215073420 2024-03-22T19:13:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63f3710b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572150d95e0 con 0x5572150d6980 2024-03-22T19:13:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63f3710b80 10 client.0 _ll_register_callbacks cb 0x7f63c4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 0x5572150d8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 0x5572150d8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60022/0 (socket says 172.21.15.86:60022) 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63ddc00700 1 -- 172.21.15.86:0/1700841584 learned_addr learned my addr 172.21.15.86:0/1700841584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.385+0000 7f63d7e00700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63ddc00700 1 -- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 msgr2=0x5572150d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63ddc00700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63ddc00700 1 -- 172.21.15.86:0/1700841584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 msgr2=0x5572150d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63dd200700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x5572150d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63ddc00700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x5572150d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63ddc00700 1 -- 172.21.15.86:0/1700841584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d810e1b0 con 0x5572150d6980 2024-03-22T19:13:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.386+0000 7f63d7e00700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.387+0000 7f63dd200700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x5572150d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.387+0000 7f63ddc00700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 0x5572150d8d70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63c8007c80 tx=0x7f63c80958d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f37cbfb7384010e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.387+0000 7f63c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.387+0000 7f63c2a00700 1 -- 172.21.15.86:0/1700841584 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8017560 con 0x5572150d6980 2024-03-22T19:13:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.388+0000 7f63c2a00700 1 -- 172.21.15.86:0/1700841584 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f63c809d2c0 con 0x5572150d6980 2024-03-22T19:13:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.388+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c8099490 con 0x5572150d6980 2024-03-22T19:13:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.388+0000 7f63c2a00700 1 -- 172.21.15.86:0/1700841584 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c80a1910 con 0x5572150d6980 2024-03-22T19:13:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.388+0000 7f63f3710b80 10 client.26041 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.389+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557214e49cd0 con 0x5572150d6980 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.389+0000 7f63c2a00700 1 -- 172.21.15.86:0/1700841584 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c8099490 con 0x5572150d6980 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63c2a00700 1 -- 172.21.15.86:0/1700841584 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f63c80c9d20 con 0x5572150d6980 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63f3710b80 10 client.26041 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63f3710b80 10 client.26041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63f3710b80 20 client.26041 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63f3710b80 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5572150daae0 0x5572150dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63f3710b80 4 client.26041 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:13:44.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.390+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557214e12cf0 con 0x5572150daae0 2024-03-22T19:13:44.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.391+0000 7f63dd200700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5572150daae0 0x5572150dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:44.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.392+0000 7f63c2a00700 10 client.26041 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:44.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.392+0000 7f63dd200700 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5572150daae0 0x5572150dcf00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19642f3e9b2340e9 server_cookie=da6e8d0236207ac5 in_seq=0 out_seq=0 2024-03-22T19:13:44.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.392+0000 7f63c2a00700 1 -- 172.21.15.86:0/1700841584 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557214e12cf0 con 0x5572150daae0 2024-03-22T19:13:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.392+0000 7f63c2a00700 10 client.26041 handle_command_reply: tid=0 2024-03-22T19:13:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.392+0000 7f63f3710b80 1 client.26041 shutdown 2024-03-22T19:13:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 msgr2=0x5572150d8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 0x5572150d8d70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63c8007c80 tx=0x7f63c80958d0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 shutdown_connections 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5572150daae0 0x5572150dcf00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572150d6980 0x5572150d8d70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63c2a00700 0 client.26041 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557215072580 0x5572150d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 --2- 172.21.15.86:0/1700841584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557215073420 0x5572150d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 >> 172.21.15.86:0/1700841584 conn(0x557214fe7220 msgr2=0x55721507a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.394+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 shutdown_connections 2024-03-22T19:13:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.395+0000 7f63f3710b80 1 -- 172.21.15.86:0/1700841584 wait complete. 2024-03-22T19:13:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.395+0000 7f63f3710b80 20 client.26041 trim_cache size 0 max 16384 2024-03-22T19:13:44.397 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:13:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f63bc0a0b10 msgr2=0x7f63bc0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f63bc0a0b10 0x7f63bc0a2fd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f63c4013a40 tx=0x7f63c400b850 comp rx=0 tx=0).stop 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108ee0 msgr2=0x7f63d81a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108ee0 0x7f63d81a4990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c970 tx=0x7f63c800cd30 comp rx=0 tx=0).stop 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 shutdown_connections 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f63bc0a0b10 0x7f63bc0a2fd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d8108ee0 0x7f63d81a4990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63d80721e0 0x7f63d819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 --2- 172.21.15.86:0/760385067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8108520 0x7f63d819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:44.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.397+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 >> 172.21.15.86:0/760385067 conn(0x7f63d806dd20 msgr2=0x7f63d810b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:44.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.398+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 shutdown_connections 2024-03-22T19:13:44.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:44.398+0000 7f63dfc00700 1 -- 172.21.15.86:0/760385067 wait complete. 2024-03-22T19:13:45.435 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:13:45.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:13:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:45 smithi086 ceph-mon[38032]: pgmap v369: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 145 MiB/s wr, 21.04k op/s 2024-03-22T19:13:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:45 smithi167 ceph-mon[47728]: pgmap v369: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 145 MiB/s wr, 21.04k op/s 2024-03-22T19:13:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 -- 172.21.15.86:0/3578877942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 msgr2=0x7f57c8101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 --2- 172.21.15.86:0/3578877942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c8101160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009990 tx=0x7f57b8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 -- 172.21.15.86:0/3578877942 shutdown_connections 2024-03-22T19:13:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 --2- 172.21.15.86:0/3578877942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 0x7f57c8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 --2- 172.21.15.86:0/3578877942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c8101160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 --2- 172.21.15.86:0/3578877942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 0x7f57c8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.921+0000 7f57cf200700 1 -- 172.21.15.86:0/3578877942 >> 172.21.15.86:0/3578877942 conn(0x7f57c80fc2f0 msgr2=0x7f57c80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:45.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.922+0000 7f57cf200700 1 -- 172.21.15.86:0/3578877942 shutdown_connections 2024-03-22T19:13:45.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.922+0000 7f57cf200700 1 -- 172.21.15.86:0/3578877942 wait complete. 2024-03-22T19:13:45.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.923+0000 7f57cf200700 1 Processor -- start 2024-03-22T19:13:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.924+0000 7f57cf200700 1 -- start start 2024-03-22T19:13:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cf200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c810f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 0x7f57c810fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cf200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 0x7f57c8116e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57c7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c810f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57c7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c810f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60036/0 (socket says 172.21.15.86:60036) 2024-03-22T19:13:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57c7e00700 1 -- 172.21.15.86:0/2994330031 learned_addr learned my addr 172.21.15.86:0/2994330031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cd200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 0x7f57c8116e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81173e0 con 0x7f57c8101f30 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8117550 con 0x7f57c81028d0 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.925+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8117710 con 0x7f57c8100d40 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57c7400700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 0x7f57c810fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57cd200700 1 -- 172.21.15.86:0/2994330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 msgr2=0x7f57c810f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57cd200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c810f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57cd200700 1 -- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 msgr2=0x7f57c810fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57cd200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 0x7f57c810fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57cd200700 1 -- 172.21.15.86:0/2994330031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80095f0 con 0x7f57c81028d0 2024-03-22T19:13:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.926+0000 7f57c7400700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 0x7f57c810fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.927+0000 7f57cd200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 0x7f57c8116e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57c000cc80 tx=0x7f57c000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.928+0000 7f57c4c00700 1 -- 172.21.15.86:0/2994330031 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000aad0 con 0x7f57c81028d0 2024-03-22T19:13:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.928+0000 7f57c4c00700 1 -- 172.21.15.86:0/2994330031 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c000ac30 con 0x7f57c81028d0 2024-03-22T19:13:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.928+0000 7f57c4c00700 1 -- 172.21.15.86:0/2994330031 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00115e0 con 0x7f57c81028d0 2024-03-22T19:13:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.928+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c81adaf0 con 0x7f57c81028d0 2024-03-22T19:13:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.928+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81adfb0 con 0x7f57c81028d0 2024-03-22T19:13:45.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.936+0000 7f57e2d48b80 1 Processor -- start 2024-03-22T19:13:45.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.936+0000 7f57c4c00700 1 -- 172.21.15.86:0/2994330031 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f57c0012420 con 0x7f57c81028d0 2024-03-22T19:13:45.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.938+0000 7f57c4c00700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57ac0a0c90 0x7f57ac0a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.938+0000 7f57c4c00700 1 -- 172.21.15.86:0/2994330031 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f57c00cbca0 con 0x7f57c81028d0 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.938+0000 7f57e2d48b80 1 -- start start 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c077585ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077586920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0774dda10 con 0x55c077585600 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0774db3d0 con 0x55c077584ca0 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0774d6cc0 con 0x55c0775864a0 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57cd200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077586920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57cd200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077586920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60040/0 (socket says 172.21.15.86:60040) 2024-03-22T19:13:45.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.939+0000 7f57cd200700 1 -- 172.21.15.86:0/771466142 learned_addr learned my addr 172.21.15.86:0/771466142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.940+0000 7f57c7e00700 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.940+0000 7f57cd200700 1 -- 172.21.15.86:0/771466142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 msgr2=0x55c0775850c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.940+0000 7f57cd200700 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775850c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.940+0000 7f57cd200700 1 -- 172.21.15.86:0/771466142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 msgr2=0x55c077585ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.940+0000 7f57cd200700 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c077585ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.940+0000 7f57cd200700 1 -- 172.21.15.86:0/771466142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c077358430 con 0x55c0775864a0 2024-03-22T19:13:45.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.941+0000 7f57c7400700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57ac0a0c90 0x7f57ac0a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.941+0000 7f57cd200700 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077586920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57c0099330 tx=0x7f57c0099b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585fc6940e4e8407 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:45.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.941+0000 7f57bd600700 1 -- 172.21.15.86:0/771466142 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00993a0 con 0x55c0775864a0 2024-03-22T19:13:45.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.942+0000 7f57bd600700 1 -- 172.21.15.86:0/771466142 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c009d1d0 con 0x55c0775864a0 2024-03-22T19:13:45.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.942+0000 7f57bd600700 1 -- 172.21.15.86:0/771466142 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00a1390 con 0x55c0775864a0 2024-03-22T19:13:45.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.943+0000 7f57c7400700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57ac0a0c90 0x7f57ac0a3150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57b4005950 tx=0x7f57b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.944+0000 7f57e2d48b80 1 -- 172.21.15.86:0/771466142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 msgr2=0x55c077586920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.944+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077586920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57c0099330 tx=0x7f57c0099b20 comp rx=0 tx=0).stop 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 -- 172.21.15.86:0/771466142 shutdown_connections 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077586920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/771466142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c077585ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 -- 172.21.15.86:0/771466142 >> 172.21.15.86:0/771466142 conn(0x55c0774fa2a0 msgr2=0x55c07757fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:45.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 -- 172.21.15.86:0/771466142 shutdown_connections 2024-03-22T19:13:45.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.945+0000 7f57e2d48b80 1 -- 172.21.15.86:0/771466142 wait complete. 2024-03-22T19:13:45.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.946+0000 7f57e2d48b80 1 -- start start 2024-03-22T19:13:45.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.946+0000 7f57e2d48b80 1 Processor -- start 2024-03-22T19:13:45.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.949+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077584ca0 0x55c07758eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.949+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 0x55c07758f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.949+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0775864a0 0x55c077596100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57c7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 0x55c07758f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57c7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 0x55c07758f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50604/0 (socket says 172.21.15.86:50604) 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57c7400700 1 -- 172.21.15.86:0/4085999697 learned_addr learned my addr 172.21.15.86:0/4085999697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57c7e00700 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077584ca0 0x55c07758eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c077472e00 con 0x55c0775864a0 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c077472aa0 con 0x55c077585600 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07735a720 con 0x55c077584ca0 2024-03-22T19:13:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.950+0000 7f57e2d48b80 10 client.0 _ll_register_callbacks cb 0x55c0772de420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57c7400700 1 -- 172.21.15.86:0/4085999697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077584ca0 msgr2=0x55c07758eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57c7400700 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077584ca0 0x55c07758eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57c7400700 1 -- 172.21.15.86:0/4085999697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0775864a0 msgr2=0x55c077596100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57c7400700 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0775864a0 0x55c077596100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57c7400700 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81adfb0 con 0x55c077585600 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57c7400700 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 0x55c07758f0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57b400ff30 tx=0x7f57b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d34c37111795a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57b2000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.951+0000 7f57b2000700 1 -- 172.21.15.86:0/4085999697 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4017040 con 0x55c077585600 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.952+0000 7f57b2000700 1 -- 172.21.15.86:0/4085999697 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f57b4003b90 con 0x55c077585600 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.952+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c077577b50 con 0x55c077585600 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.952+0000 7f57b2000700 1 -- 172.21.15.86:0/4085999697 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b4003d50 con 0x55c077585600 2024-03-22T19:13:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.953+0000 7f57e2d48b80 10 client.26057 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.953+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c077304be0 con 0x55c077585600 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.953+0000 7f57b2000700 1 -- 172.21.15.86:0/4085999697 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c077577b50 con 0x55c077585600 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.954+0000 7f57b2000700 1 -- 172.21.15.86:0/4085999697 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f57b401d040 con 0x55c077585600 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.954+0000 7f57e2d48b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.954+0000 7f57e2d48b80 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.954+0000 7f57e2d48b80 20 client.26057 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.954+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c07759aa90 0x55c07759ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.955+0000 7f57e2d48b80 4 client.26057 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.955+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0774cf780 con 0x55c07759aa90 2024-03-22T19:13:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.955+0000 7f57c7e00700 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c07759aa90 0x55c07759ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:45.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.956+0000 7f57c7e00700 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c07759aa90 0x55c07759ceb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cd95df611fbda8 server_cookie=f3df701f54fa53e5 in_seq=0 out_seq=0 2024-03-22T19:13:45.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.956+0000 7f57b2000700 10 client.26057 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.957+0000 7f57b2000700 1 -- 172.21.15.86:0/4085999697 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c0774cf780 con 0x55c07759aa90 2024-03-22T19:13:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.957+0000 7f57b2000700 10 client.26057 handle_command_reply: tid=0 2024-03-22T19:13:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.957+0000 7f57e2d48b80 1 client.26057 shutdown 2024-03-22T19:13:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 msgr2=0x55c07758f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 0x55c07758f0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57b400ff30 tx=0x7f57b4004750 comp rx=0 tx=0).stop 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 shutdown_connections 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c07759aa90 0x55c07759ceb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0775864a0 0x55c077596100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077584ca0 0x55c07758eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/4085999697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077585600 0x55c07758f0c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 >> 172.21.15.86:0/4085999697 conn(0x55c0774fa2a0 msgr2=0x55c07757f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.958+0000 7f57b2000700 0 client.26057 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.959+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 shutdown_connections 2024-03-22T19:13:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.959+0000 7f57e2d48b80 1 -- 172.21.15.86:0/4085999697 wait complete. 2024-03-22T19:13:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.961+0000 7f57e2d48b80 20 client.26057 trim_cache size 0 max 16384 2024-03-22T19:13:45.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:45 smithi131 ceph-mon[78559]: pgmap v369: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 145 MiB/s wr, 21.04k op/s 2024-03-22T19:13:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:45.998+0000 7f57e2d48b80 1 Processor -- start 2024-03-22T19:13:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 -- start start 2024-03-22T19:13:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c0775893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077589900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07758d020 con 0x55c077585600 2024-03-22T19:13:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07758e6f0 con 0x55c077584ca0 2024-03-22T19:13:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.000+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0775809b0 con 0x55c0775864a0 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.001+0000 7f57c7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.001+0000 7f57cd200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077589900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.001+0000 7f57c7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50612/0 (socket says 172.21.15.86:50612) 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.001+0000 7f57c7e00700 1 -- 172.21.15.86:0/587717263 learned_addr learned my addr 172.21.15.86:0/587717263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.001+0000 7f57c7400700 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c0775893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.002+0000 7f57c7e00700 1 -- 172.21.15.86:0/587717263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 msgr2=0x55c077589900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.002+0000 7f57c7e00700 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077589900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.002+0000 7f57c7e00700 1 -- 172.21.15.86:0/587717263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 msgr2=0x55c0775893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.002+0000 7f57c7e00700 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c0775893c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.002+0000 7f57c7e00700 1 -- 172.21.15.86:0/587717263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07749cf20 con 0x55c077584ca0 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.002+0000 7f57c7e00700 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775940b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b8019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd678327583f8c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.003+0000 7f57be000700 1 -- 172.21.15.86:0/587717263 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b801d040 con 0x55c077584ca0 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.003+0000 7f57be000700 1 -- 172.21.15.86:0/587717263 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b801e040 con 0x55c077584ca0 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.003+0000 7f57be000700 1 -- 172.21.15.86:0/587717263 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8028bc0 con 0x55c077584ca0 2024-03-22T19:13:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 -- 172.21.15.86:0/587717263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 msgr2=0x55c0775940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775940b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b8019040 comp rx=0 tx=0).stop 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 -- 172.21.15.86:0/587717263 shutdown_connections 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0775864a0 0x55c077589900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c077584ca0 0x55c0775940b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/587717263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077585600 0x55c0775893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 -- 172.21.15.86:0/587717263 >> 172.21.15.86:0/587717263 conn(0x55c0774fa2a0 msgr2=0x55c07758dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 -- 172.21.15.86:0/587717263 shutdown_connections 2024-03-22T19:13:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.005+0000 7f57e2d48b80 1 -- 172.21.15.86:0/587717263 wait complete. 2024-03-22T19:13:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.006+0000 7f57e2d48b80 1 -- start start 2024-03-22T19:13:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.006+0000 7f57e2d48b80 1 Processor -- start 2024-03-22T19:13:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.007+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077584ca0 0x55c077592780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.007+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 0x55c077592cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57e2d48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0775864a0 0x55c077593200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c077594400 con 0x55c077584ca0 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0775ec5a0 con 0x55c0775864a0 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57e2d48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0775ec6e0 con 0x55c077585600 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57e2d48b80 10 client.0 _ll_register_callbacks cb 0x7f57b400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077584ca0 0x55c077592780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57c7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 0x55c077592cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57c7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 0x55c077592cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60076/0 (socket says 172.21.15.86:60076) 2024-03-22T19:13:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57c7400700 1 -- 172.21.15.86:0/1714111819 learned_addr learned my addr 172.21.15.86:0/1714111819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.008+0000 7f57cd200700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0775864a0 0x55c077593200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7400700 1 -- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0775864a0 msgr2=0x55c077593200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7400700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0775864a0 0x55c077593200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7400700 1 -- 172.21.15.86:0/1714111819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077584ca0 msgr2=0x55c077592780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7400700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077584ca0 0x55c077592780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7400700 1 -- 172.21.15.86:0/1714111819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c8109ad0 con 0x55c077585600 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7e00700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077584ca0 0x55c077592780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57cd200700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0775864a0 0x55c077593200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57c7400700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 0x55c077592cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57b400f9d0 tx=0x7f57b401f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9029cadd10ac42 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57b2000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.009+0000 7f57b2000700 1 -- 172.21.15.86:0/1714111819 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4029ea0 con 0x55c077585600 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.010+0000 7f57b2000700 1 -- 172.21.15.86:0/1714111819 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f57b4003990 con 0x55c077585600 2024-03-22T19:13:46.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.010+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c009d1d0 con 0x55c077585600 2024-03-22T19:13:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.012+0000 7f57b2000700 1 -- 172.21.15.86:0/1714111819 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b4003b50 con 0x55c077585600 2024-03-22T19:13:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.012+0000 7f57e2d48b80 10 client.26059 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.012+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c07735cd50 con 0x55c077585600 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.013+0000 7f57b2000700 1 -- 172.21.15.86:0/1714111819 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c009d1d0 con 0x55c077585600 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57b2000700 1 -- 172.21.15.86:0/1714111819 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f57b4004030 con 0x55c077585600 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57e2d48b80 10 client.26059 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57e2d48b80 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57e2d48b80 20 client.26059 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c0775ee200 0x55c0775f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57e2d48b80 4 client.26059 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:13:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.015+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c077325d70 con 0x55c0775ee200 2024-03-22T19:13:46.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.017+0000 7f57c7e00700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c0775ee200 0x55c0775f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.019+0000 7f57c7e00700 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c0775ee200 0x55c0775f0620 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5047a46ddf38f4 server_cookie=9a77723945fcdb1c in_seq=0 out_seq=0 2024-03-22T19:13:46.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.019+0000 7f57b2000700 10 client.26059 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57b2000700 1 -- 172.21.15.86:0/1714111819 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c077325d70 con 0x55c0775ee200 2024-03-22T19:13:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57b2000700 10 client.26059 handle_command_reply: tid=0 2024-03-22T19:13:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 client.26059 shutdown 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 msgr2=0x55c077592cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 0x55c077592cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57b400f9d0 tx=0x7f57b401f740 comp rx=0 tx=0).stop 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 shutdown_connections 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55c0775ee200 0x55c0775f0620 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0775864a0 0x55c077593200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c077584ca0 0x55c077592780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 --2- 172.21.15.86:0/1714111819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c077585600 0x55c077592cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.020+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 >> 172.21.15.86:0/1714111819 conn(0x55c0774fa2a0 msgr2=0x55c0775893c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.021+0000 7f57b2000700 0 client.26059 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:46.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.021+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 shutdown_connections 2024-03-22T19:13:46.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.021+0000 7f57e2d48b80 1 -- 172.21.15.86:0/1714111819 wait complete. 2024-03-22T19:13:46.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.021+0000 7f57e2d48b80 20 client.26059 trim_cache size 0 max 16384 2024-03-22T19:13:46.023 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:13:46.023 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e34a3ccd-deee-4f94-9659-5669e603b7be", 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.022+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57ac0a0c90 msgr2=0x7f57ac0a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.022+0000 7f57cf200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57ac0a0c90 0x7f57ac0a3150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57b4005950 tx=0x7f57b40058e0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.022+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 msgr2=0x7f57c8116e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.022+0000 7f57cf200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 0x7f57c8116e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57c000cc80 tx=0x7f57c000cf90 comp rx=0 tx=0).stop 2024-03-22T19:13:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.023+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 shutdown_connections 2024-03-22T19:13:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.023+0000 7f57cf200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f57ac0a0c90 0x7f57ac0a3150 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.023+0000 7f57cf200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57c81028d0 0x7f57c8116e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.023+0000 7f57cf200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c8100d40 0x7f57c810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.023+0000 7f57cf200700 1 --2- 172.21.15.86:0/2994330031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c8101f30 0x7f57c810fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.023+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 >> 172.21.15.86:0/2994330031 conn(0x7f57c80fc2f0 msgr2=0x7f57c810d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.025+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 shutdown_connections 2024-03-22T19:13:46.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.025+0000 7f57cf200700 1 -- 172.21.15.86:0/2994330031 wait complete. 2024-03-22T19:13:46.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:13:46.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 -- 172.21.15.86:0/3795894011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc107350 msgr2=0x7fb3cc107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/3795894011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc107350 0x7fb3cc107770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009990 tx=0x7fb3bc009ca0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 -- 172.21.15.86:0/3795894011 shutdown_connections 2024-03-22T19:13:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/3795894011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3cc108ee0 0x7fb3cc10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/3795894011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc107350 0x7fb3cc107770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/3795894011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc108540 0x7fb3cc1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 -- 172.21.15.86:0/3795894011 >> 172.21.15.86:0/3795894011 conn(0x7fb3cc076320 msgr2=0x7fb3cc078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.524+0000 7fb3d2c00700 1 -- 172.21.15.86:0/3795894011 shutdown_connections 2024-03-22T19:13:46.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.525+0000 7fb3d2c00700 1 -- 172.21.15.86:0/3795894011 wait complete. 2024-03-22T19:13:46.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.526+0000 7fb3d2c00700 1 Processor -- start 2024-03-22T19:13:46.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.526+0000 7fb3d2c00700 1 -- start start 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3d2c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc107350 0x7fb3cc07e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3d2c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 0x7fb3cc07e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3d2c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3cc108ee0 0x7fb3cc1ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3cb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 0x7fb3cc07e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3cb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 0x7fb3cc07e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60094/0 (socket says 172.21.15.86:60094) 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc107350 0x7fb3cc07e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.527+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc107350 0x7fb3cc07e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50650/0 (socket says 172.21.15.86:50650) 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3d2c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc081a80 con 0x7fb3cc108ee0 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3d2c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc081bf0 con 0x7fb3cc107350 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3d2c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc1ada50 con 0x7fb3cc108540 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3cb400700 1 -- 172.21.15.86:0/4103013938 learned_addr learned my addr 172.21.15.86:0/4103013938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3cb400700 1 -- 172.21.15.86:0/4103013938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc107350 msgr2=0x7fb3cc07e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3cb400700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc107350 0x7fb3cc07e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3cb400700 1 -- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3cc108ee0 msgr2=0x7fb3cc1ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3cb400700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3cc108ee0 0x7fb3cc1ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.529+0000 7fb3cb400700 1 -- 172.21.15.86:0/4103013938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc0095f0 con 0x7fb3cc108540 2024-03-22T19:13:46.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.530+0000 7fb3cb400700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 0x7fb3cc07e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ada0 tx=0x7fb3c400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.531+0000 7fb3c8c00700 1 -- 172.21.15.86:0/4103013938 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f070 con 0x7fb3cc108540 2024-03-22T19:13:46.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.531+0000 7fb3c8c00700 1 -- 172.21.15.86:0/4103013938 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c400ae90 con 0x7fb3cc108540 2024-03-22T19:13:46.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.531+0000 7fb3c8c00700 1 -- 172.21.15.86:0/4103013938 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f070 con 0x7fb3cc108540 2024-03-22T19:13:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.532+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc1adc50 con 0x7fb3cc108540 2024-03-22T19:13:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.532+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc1ae080 con 0x7fb3cc108540 2024-03-22T19:13:46.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.537+0000 7fb3c8c00700 1 -- 172.21.15.86:0/4103013938 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fb3c4004ad0 con 0x7fb3cc108540 2024-03-22T19:13:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.540+0000 7fb3c8c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3b00a0b90 0x7fb3b00a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.541+0000 7fb3e66f6b80 1 Processor -- start 2024-03-22T19:13:46.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3c8c00700 1 -- 172.21.15.86:0/4103013938 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb3c40caf30 con 0x7fb3cc108540 2024-03-22T19:13:46.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 -- start start 2024-03-22T19:13:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de71d990 con 0x5562de7c6420 2024-03-22T19:13:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de71b350 con 0x5562de7c4c20 2024-03-22T19:13:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.542+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de716c40 con 0x5562de7c5580 2024-03-22T19:13:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.544+0000 7fb3cb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.544+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3b00a0b90 0x7fb3b00a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.544+0000 7fb3cb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60100/0 (socket says 172.21.15.86:60100) 2024-03-22T19:13:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.544+0000 7fb3cb400700 1 -- 172.21.15.86:0/1985472740 learned_addr learned my addr 172.21.15.86:0/1985472740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.545+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.545+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.545+0000 7fb3cb400700 1 -- 172.21.15.86:0/1985472740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 msgr2=0x5562de7c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.545+0000 7fb3cb400700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.546+0000 7fb3cb400700 1 -- 172.21.15.86:0/1985472740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 msgr2=0x5562de7c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.546+0000 7fb3cb400700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.546+0000 7fb3cb400700 1 -- 172.21.15.86:0/1985472740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562de5983b0 con 0x5562de7c5580 2024-03-22T19:13:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.546+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.546+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:13:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.547+0000 7fb3cb400700 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c5e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40950c0 tx=0x7fb3c40984a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2545ead20d60003c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.551+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3b00a0b90 0x7fb3b00a3050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0038c0 tx=0x7fb3bc009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.555+0000 7fb3c1200700 1 -- 172.21.15.86:0/1985472740 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f070 con 0x5562de7c5580 2024-03-22T19:13:46.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.555+0000 7fb3c1200700 1 -- 172.21.15.86:0/1985472740 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c4098d50 con 0x5562de7c5580 2024-03-22T19:13:46.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.555+0000 7fb3c1200700 1 -- 172.21.15.86:0/1985472740 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4096790 con 0x5562de7c5580 2024-03-22T19:13:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/1985472740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 msgr2=0x5562de7c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c5e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40950c0 tx=0x7fb3c40984a0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/1985472740 shutdown_connections 2024-03-22T19:13:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/1985472740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c5e40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.556+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/1985472740 >> 172.21.15.86:0/1985472740 conn(0x5562de73a220 msgr2=0x5562de7bfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.557+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/1985472740 shutdown_connections 2024-03-22T19:13:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.557+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/1985472740 wait complete. 2024-03-22T19:13:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.558+0000 7fb3e66f6b80 1 -- start start 2024-03-22T19:13:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.559+0000 7fb3e66f6b80 1 Processor -- start 2024-03-22T19:13:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de6b2d80 con 0x5562de7c6420 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de6b2a20 con 0x5562de7c4c20 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de59a6a0 con 0x5562de7c5580 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.561+0000 7fb3e66f6b80 10 client.0 _ll_register_callbacks cb 0x5562de51e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.562+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.562+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:50678/0 (socket says 172.21.15.86:50678) 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.562+0000 7fb3cbe00700 1 -- 172.21.15.86:0/4044835933 learned_addr learned my addr 172.21.15.86:0/4044835933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.563+0000 7fb3cb400700 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.564+0000 7fb3cbe00700 1 -- 172.21.15.86:0/4044835933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 msgr2=0x5562de7ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.564+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.564+0000 7fb3cbe00700 1 -- 172.21.15.86:0/4044835933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 msgr2=0x5562de7d6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.564+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.564+0000 7fb3cbe00700 1 -- 172.21.15.86:0/4044835933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc1ae080 con 0x5562de7c4c20 2024-03-22T19:13:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.566+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7ceab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc015000 tx=0x7fb3bc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a510a82d3cfd894 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.566+0000 7fb3b9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.567+0000 7fb3b9600700 1 -- 172.21.15.86:0/4044835933 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc03d070 con 0x5562de7c4c20 2024-03-22T19:13:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.567+0000 7fb3b9600700 1 -- 172.21.15.86:0/4044835933 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb3bc003e10 con 0x5562de7c4c20 2024-03-22T19:13:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.568+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562de7b7ad0 con 0x5562de7c4c20 2024-03-22T19:13:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.572+0000 7fb3b9600700 1 -- 172.21.15.86:0/4044835933 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3bc036650 con 0x5562de7c4c20 2024-03-22T19:13:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.572+0000 7fb3e66f6b80 10 client.26069 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.572+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5562de544b60 con 0x5562de7c4c20 2024-03-22T19:13:46.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3b9600700 1 -- 172.21.15.86:0/4044835933 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3bc049030 con 0x5562de7c4c20 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3b9600700 1 -- 172.21.15.86:0/4044835933 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb3bc042870 con 0x5562de7c4c20 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3e66f6b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3e66f6b80 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3e66f6b80 20 client.26069 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de7da830 0x5562de7dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3e66f6b80 4 client.26069 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562de70f700 con 0x5562de7da830 2024-03-22T19:13:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.573+0000 7fb3cb400700 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de7da830 0x5562de7dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.575+0000 7fb3cb400700 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de7da830 0x5562de7dcc50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aad141d8753a883 server_cookie=ae530eb9e48e2fc5 in_seq=0 out_seq=0 2024-03-22T19:13:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.575+0000 7fb3b9600700 10 client.26069 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:46.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.577+0000 7fb3b9600700 1 -- 172.21.15.86:0/4044835933 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5562de70f700 con 0x5562de7da830 2024-03-22T19:13:46.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.577+0000 7fb3b9600700 10 client.26069 handle_command_reply: tid=0 2024-03-22T19:13:46.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.578+0000 7fb3e66f6b80 1 client.26069 shutdown 2024-03-22T19:13:46.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.579+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 msgr2=0x5562de7ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.579+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7ceab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc015000 tx=0x7fb3bc009310 comp rx=0 tx=0).stop 2024-03-22T19:13:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.579+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 shutdown_connections 2024-03-22T19:13:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.579+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de7da830 0x5562de7dcc50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3b9600700 0 client.26069 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:46.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c6420 0x5562de7d6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c4c20 0x5562de7ceab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/4044835933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 >> 172.21.15.86:0/4044835933 conn(0x5562de73a220 msgr2=0x5562de7bf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 shutdown_connections 2024-03-22T19:13:46.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.581+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/4044835933 wait complete. 2024-03-22T19:13:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.583+0000 7fb3e66f6b80 20 client.26069 trim_cache size 0 max 16384 2024-03-22T19:13:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.639+0000 7fb3e66f6b80 1 Processor -- start 2024-03-22T19:13:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.640+0000 7fb3e66f6b80 1 -- start start 2024-03-22T19:13:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.641+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de7d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.641+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.641+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de7cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.641+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de7bc5e0 con 0x5562de7c4c20 2024-03-22T19:13:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.641+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de7d71b0 con 0x5562de7c6420 2024-03-22T19:13:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.641+0000 7fb3e66f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de7b7610 con 0x5562de7c5580 2024-03-22T19:13:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.642+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de7d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.642+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de7d3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39092/0 (socket says 172.21.15.86:39092) 2024-03-22T19:13:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.642+0000 7fb3cbe00700 1 -- 172.21.15.86:0/672697037 learned_addr learned my addr 172.21.15.86:0/672697037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.642+0000 7fb3cb400700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.642+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de7cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cbe00700 1 -- 172.21.15.86:0/672697037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 msgr2=0x5562de7c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cbe00700 1 -- 172.21.15.86:0/672697037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 msgr2=0x5562de7cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de7cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cbe00700 1 -- 172.21.15.86:0/672697037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562de6dcea0 con 0x5562de7c4c20 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cb400700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de7cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.645+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de7d3fb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0098e0 tx=0x7fb3bc046f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bf38fecf764cd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.646+0000 7fb3c1c00700 1 -- 172.21.15.86:0/672697037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc03d070 con 0x5562de7c4c20 2024-03-22T19:13:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.646+0000 7fb3c1c00700 1 -- 172.21.15.86:0/672697037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3bc049070 con 0x5562de7c4c20 2024-03-22T19:13:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.646+0000 7fb3c1c00700 1 -- 172.21.15.86:0/672697037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc036930 con 0x5562de7c4c20 2024-03-22T19:13:46.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/672697037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 msgr2=0x5562de7d3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de7d3fb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0098e0 tx=0x7fb3bc046f00 comp rx=0 tx=0).stop 2024-03-22T19:13:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/672697037 shutdown_connections 2024-03-22T19:13:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de7cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de7d3fb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/672697037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de7c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.647+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/672697037 >> 172.21.15.86:0/672697037 conn(0x5562de73a220 msgr2=0x5562de7ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.648+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/672697037 shutdown_connections 2024-03-22T19:13:46.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.648+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/672697037 wait complete. 2024-03-22T19:13:46.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.648+0000 7fb3e66f6b80 1 -- start start 2024-03-22T19:13:46.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.649+0000 7fb3e66f6b80 1 Processor -- start 2024-03-22T19:13:46.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.652+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de824530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.652+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de824a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.652+0000 7fb3e66f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de82b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.652+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de824530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de824530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39094/0 (socket says 172.21.15.86:39094) 2024-03-22T19:13:46.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3cb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de824a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3cbe00700 1 -- 172.21.15.86:0/2346752808 learned_addr learned my addr 172.21.15.86:0/2346752808 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:13:46.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de82bab0 con 0x5562de7c4c20 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de82bbf0 con 0x5562de7c6420 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562de82bdb0 con 0x5562de7c5580 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.653+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de82b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.654+0000 7fb3e66f6b80 10 client.0 _ll_register_callbacks cb 0x7fb3bc003c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.654+0000 7fb3d0c00700 1 -- 172.21.15.86:0/2346752808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 msgr2=0x5562de824a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.654+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de824a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.654+0000 7fb3d0c00700 1 -- 172.21.15.86:0/2346752808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 msgr2=0x5562de824530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.654+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de824530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.654+0000 7fb3d0c00700 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562de544b60 con 0x5562de7c6420 2024-03-22T19:13:46.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.655+0000 7fb3cb400700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de824a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:13:46.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.655+0000 7fb3d0c00700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de82b570 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5562de7c5c00 tx=0x7fb3b400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25b2adec54a6b42f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:13:46.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.655+0000 7fb3b9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:13:46.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.655+0000 7fb3b9600700 1 -- 172.21.15.86:0/2346752808 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400ee30 con 0x5562de7c6420 2024-03-22T19:13:46.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.656+0000 7fb3b9600700 1 -- 172.21.15.86:0/2346752808 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb3b4013450 con 0x5562de7c6420 2024-03-22T19:13:46.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.656+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3c4098d50 con 0x5562de7c6420 2024-03-22T19:13:46.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.656+0000 7fb3b9600700 1 -- 172.21.15.86:0/2346752808 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b4013610 con 0x5562de7c6420 2024-03-22T19:13:46.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.656+0000 7fb3e66f6b80 10 client.26075 fetch_fsmap learned FSMap version 32 2024-03-22T19:13:46.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.656+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5562de59ccd0 con 0x5562de7c6420 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.657+0000 7fb3b9600700 1 -- 172.21.15.86:0/2346752808 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c4098d50 con 0x5562de7c6420 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.657+0000 7fb3b9600700 1 -- 172.21.15.86:0/2346752808 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb3b4011070 con 0x5562de7c6420 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.657+0000 7fb3e66f6b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 32 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.657+0000 7fb3e66f6b80 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.657+0000 7fb3e66f6b80 20 client.26075 populate_metadata read hostname 'smithi086' 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.657+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de82d8d0 0x5562de82fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:13:46.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.658+0000 7fb3e66f6b80 4 client.26075 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:13:46.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.658+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5562de565cf0 con 0x5562de82d8d0 2024-03-22T19:13:46.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.658+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de82d8d0 0x5562de82fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:13:46.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3b9600700 10 client.26075 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:46.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3cbe00700 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de82d8d0 0x5562de82fcf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623f2107b4280fb0 server_cookie=cf52408b04d12e6 in_seq=0 out_seq=0 2024-03-22T19:13:46.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3b9600700 1 -- 172.21.15.86:0/2346752808 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5562de565cf0 con 0x5562de82d8d0 2024-03-22T19:13:46.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3b9600700 10 client.26075 handle_command_reply: tid=0 2024-03-22T19:13:46.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3e66f6b80 1 client.26075 shutdown 2024-03-22T19:13:46.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 msgr2=0x5562de82b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.659+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de82b570 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5562de7c5c00 tx=0x7fb3b400cf90 comp rx=0 tx=0).stop 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 shutdown_connections 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5562de82d8d0 0x5562de82fcf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562de7c6420 0x5562de82b570 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562de7c4c20 0x5562de824530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3b9600700 0 client.26075 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 --2- 172.21.15.86:0/2346752808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562de7c5580 0x5562de824a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 >> 172.21.15.86:0/2346752808 conn(0x5562de73a220 msgr2=0x5562de7cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.660+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 shutdown_connections 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.661+0000 7fb3e66f6b80 1 -- 172.21.15.86:0/2346752808 wait complete. 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.661+0000 7fb3e66f6b80 20 client.26075 trim_cache size 0 max 16384 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (91 inodes in the stack)", 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "e34a3ccd-deee-4f94-9659-5669e603b7be": { 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:13:46.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e34a3ccd-deee-4f94-9659-5669e603b7be", 2024-03-22T19:13:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:13:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:13:46.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:13:46.664 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3b00a0b90 msgr2=0x7fb3b00a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3b00a0b90 0x7fb3b00a3050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0038c0 tx=0x7fb3bc009f00 comp rx=0 tx=0).stop 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 msgr2=0x7fb3cc07e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 0x7fb3cc07e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ada0 tx=0x7fb3c400c330 comp rx=0 tx=0).stop 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 shutdown_connections 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb3b00a0b90 0x7fb3b00a3050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3cc108ee0 0x7fb3cc1ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3cc107350 0x7fb3cc07e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 --2- 172.21.15.86:0/4103013938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3cc108540 0x7fb3cc07e9f0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ada0 tx=0x7fb3c400c330 comp rx=0 tx=0).stop 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.662+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 >> 172.21.15.86:0/4103013938 conn(0x7fb3cc076320 msgr2=0x7fb3cc10b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.663+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 shutdown_connections 2024-03-22T19:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:13:46.663+0000 7fb3d2c00700 1 -- 172.21.15.86:0/4103013938 wait complete. 2024-03-22T19:13:46.701 INFO:tasks.cephfs.filesystem:scrub status for tag:e34a3ccd-deee-4f94-9659-5669e603b7be - {'path': '/', 'tag': 'e34a3ccd-deee-4f94-9659-5669e603b7be', 'options': 'recursive,force'} 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:47 smithi086 ceph-mon[38032]: pgmap v370: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 158 MiB/s wr, 23.00k op/s 2024-03-22T19:13:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:13:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:47.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:47.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:13:47.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:47 smithi167 ceph-mon[47728]: pgmap v370: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 158 MiB/s wr, 23.00k op/s 2024-03-22T19:13:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:13:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:13:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:13:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:47 smithi131 ceph-mon[78559]: pgmap v370: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 158 MiB/s wr, 23.00k op/s 2024-03-22T19:13:49.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:49 smithi167 ceph-mon[47728]: pgmap v371: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 167 MiB/s wr, 23.18k op/s 2024-03-22T19:13:49.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:49 smithi131 ceph-mon[78559]: pgmap v371: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 167 MiB/s wr, 23.18k op/s 2024-03-22T19:13:49.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:49 smithi086 ceph-mon[38032]: pgmap v371: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 167 MiB/s wr, 23.18k op/s 2024-03-22T19:13:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:51 smithi131 ceph-mon[78559]: pgmap v372: 97 pgs: 97 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 155 MiB/s wr, 20.55k op/s 2024-03-22T19:13:51.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:51 smithi086 ceph-mon[38032]: pgmap v372: 97 pgs: 97 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 155 MiB/s wr, 20.55k op/s 2024-03-22T19:13:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:51 smithi167 ceph-mon[47728]: pgmap v372: 97 pgs: 97 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 155 MiB/s wr, 20.55k op/s 2024-03-22T19:13:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:53 smithi131 ceph-mon[78559]: pgmap v373: 97 pgs: 97 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 170 MiB/s wr, 21.69k op/s 2024-03-22T19:13:53.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:53 smithi086 ceph-mon[38032]: pgmap v373: 97 pgs: 97 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 170 MiB/s wr, 21.69k op/s 2024-03-22T19:13:53.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:53 smithi167 ceph-mon[47728]: pgmap v373: 97 pgs: 97 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 170 MiB/s wr, 21.69k op/s 2024-03-22T19:13:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:55 smithi131 ceph-mon[78559]: pgmap v374: 97 pgs: 97 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 161 MiB/s wr, 19.75k op/s 2024-03-22T19:13:55.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:55 smithi086 ceph-mon[38032]: pgmap v374: 97 pgs: 97 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 161 MiB/s wr, 19.75k op/s 2024-03-22T19:13:55.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:55 smithi167 ceph-mon[47728]: pgmap v374: 97 pgs: 97 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 161 MiB/s wr, 19.75k op/s 2024-03-22T19:13:57.508 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:57 smithi086 ceph-mon[38032]: pgmap v375: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 174 MiB/s wr, 20.88k op/s 2024-03-22T19:13:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:57 smithi167 ceph-mon[47728]: pgmap v375: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 174 MiB/s wr, 20.88k op/s 2024-03-22T19:13:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:57 smithi131 ceph-mon[78559]: pgmap v375: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 174 MiB/s wr, 20.88k op/s 2024-03-22T19:13:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:13:59 smithi167 ceph-mon[47728]: pgmap v376: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 185 MiB/s wr, 22.15k op/s 2024-03-22T19:13:59.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:13:59 smithi131 ceph-mon[78559]: pgmap v376: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 185 MiB/s wr, 22.15k op/s 2024-03-22T19:13:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:13:59 smithi086 ceph-mon[38032]: pgmap v376: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 185 MiB/s wr, 22.15k op/s 2024-03-22T19:14:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:01 smithi131 ceph-mon[78559]: pgmap v377: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 176 MiB/s wr, 21.13k op/s 2024-03-22T19:14:01.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:01 smithi086 ceph-mon[38032]: pgmap v377: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 176 MiB/s wr, 21.13k op/s 2024-03-22T19:14:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:01 smithi167 ceph-mon[47728]: pgmap v377: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 176 MiB/s wr, 21.13k op/s 2024-03-22T19:14:03.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:03 smithi131 ceph-mon[78559]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 194 MiB/s wr, 23.62k op/s 2024-03-22T19:14:03.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:03 smithi086 ceph-mon[38032]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 194 MiB/s wr, 23.62k op/s 2024-03-22T19:14:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:03 smithi167 ceph-mon[47728]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 194 MiB/s wr, 23.62k op/s 2024-03-22T19:14:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:05 smithi131 ceph-mon[78559]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 181 MiB/s wr, 22.23k op/s 2024-03-22T19:14:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:05 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:14:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:05 smithi086 ceph-mon[38032]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 181 MiB/s wr, 22.23k op/s 2024-03-22T19:14:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:05 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:14:05.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:05 smithi167 ceph-mon[47728]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 181 MiB/s wr, 22.23k op/s 2024-03-22T19:14:05.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:05 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:14:07.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:07 smithi086 ceph-mon[38032]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 174 MiB/s wr, 21.80k op/s 2024-03-22T19:14:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:07 smithi167 ceph-mon[47728]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 174 MiB/s wr, 21.80k op/s 2024-03-22T19:14:07.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:07 smithi131 ceph-mon[78559]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 174 MiB/s wr, 21.80k op/s 2024-03-22T19:14:09.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:09 smithi131 ceph-mon[78559]: pgmap v381: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 164 MiB/s wr, 20.66k op/s 2024-03-22T19:14:09.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:09 smithi086 ceph-mon[38032]: pgmap v381: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 164 MiB/s wr, 20.66k op/s 2024-03-22T19:14:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:09 smithi167 ceph-mon[47728]: pgmap v381: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 164 MiB/s wr, 20.66k op/s 2024-03-22T19:14:11.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:11 smithi131 ceph-mon[78559]: pgmap v382: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 158 MiB/s wr, 19.91k op/s 2024-03-22T19:14:11.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:11 smithi086 ceph-mon[38032]: pgmap v382: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 158 MiB/s wr, 19.91k op/s 2024-03-22T19:14:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:11 smithi167 ceph-mon[47728]: pgmap v382: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 158 MiB/s wr, 19.91k op/s 2024-03-22T19:14:13.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:13 smithi131 ceph-mon[78559]: pgmap v383: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 170 MiB/s wr, 21.79k op/s 2024-03-22T19:14:13.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:13 smithi086 ceph-mon[38032]: pgmap v383: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 170 MiB/s wr, 21.79k op/s 2024-03-22T19:14:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:13 smithi167 ceph-mon[47728]: pgmap v383: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 170 MiB/s wr, 21.79k op/s 2024-03-22T19:14:15.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:15 smithi131 ceph-mon[78559]: pgmap v384: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 135 MiB/s wr, 17.27k op/s 2024-03-22T19:14:15.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:15 smithi086 ceph-mon[38032]: pgmap v384: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 135 MiB/s wr, 17.27k op/s 2024-03-22T19:14:15.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:15 smithi167 ceph-mon[47728]: pgmap v384: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 135 MiB/s wr, 17.27k op/s 2024-03-22T19:14:16.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:14:16.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:16 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:14:16.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:16 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:14:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:16 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:14:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.082+0000 7efe0e400700 1 -- 172.21.15.86:0/2799856806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 msgr2=0x7efe0810cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.082+0000 7efe0e400700 1 --2- 172.21.15.86:0/2799856806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 0x7efe0810cbc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efdf40074e0 tx=0x7efdf400af90 comp rx=0 tx=0).stop 2024-03-22T19:14:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.083+0000 7efe0e400700 1 -- 172.21.15.86:0/2799856806 shutdown_connections 2024-03-22T19:14:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.083+0000 7efe0e400700 1 --2- 172.21.15.86:0/2799856806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0810d250 0x7efe0810f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.083+0000 7efe0e400700 1 --2- 172.21.15.86:0/2799856806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe08107db0 0x7efe0810a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.083+0000 7efe0e400700 1 --2- 172.21.15.86:0/2799856806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 0x7efe0810cbc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.083+0000 7efe0e400700 1 -- 172.21.15.86:0/2799856806 >> 172.21.15.86:0/2799856806 conn(0x7efe0806da20 msgr2=0x7efe0806fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.085+0000 7efe0e400700 1 -- 172.21.15.86:0/2799856806 shutdown_connections 2024-03-22T19:14:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.085+0000 7efe0e400700 1 -- 172.21.15.86:0/2799856806 wait complete. 2024-03-22T19:14:17.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.087+0000 7efe0e400700 1 Processor -- start 2024-03-22T19:14:17.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.087+0000 7efe0e400700 1 -- start start 2024-03-22T19:14:17.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.088+0000 7efe0e400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 0x7efe081a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.088+0000 7efe0e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 0x7efe081a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.088+0000 7efe0e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0810d250 0x7efe081a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.088+0000 7efe0e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe081a9190 con 0x7efe0810d250 2024-03-22T19:14:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.088+0000 7efe0e400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe081a92d0 con 0x7efe0810a770 2024-03-22T19:14:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.088+0000 7efe0e400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe081a9460 con 0x7efe08107db0 2024-03-22T19:14:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.089+0000 7efe0d000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 0x7efe081a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.089+0000 7efe0d000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 0x7efe081a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:56872/0 (socket says 172.21.15.86:56872) 2024-03-22T19:14:17.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.089+0000 7efe0d000700 1 -- 172.21.15.86:0/505007213 learned_addr learned my addr 172.21.15.86:0/505007213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:17.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.090+0000 7efe0da00700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0810d250 0x7efe081a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.091+0000 7efe07e00700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 0x7efe081a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.091+0000 7efe0d000700 1 -- 172.21.15.86:0/505007213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 msgr2=0x7efe081a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.091+0000 7efe0d000700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 0x7efe081a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.091+0000 7efe0d000700 1 -- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0810d250 msgr2=0x7efe081a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.091+0000 7efe0d000700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0810d250 0x7efe081a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.091+0000 7efe0d000700 1 -- 172.21.15.86:0/505007213 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40071c0 con 0x7efe08107db0 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.092+0000 7efe0d000700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 0x7efe081a16b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe0000e9a0 tx=0x7efe0000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.093+0000 7efe05600700 1 -- 172.21.15.86:0/505007213 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0000cbd0 con 0x7efe08107db0 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.093+0000 7efe05600700 1 -- 172.21.15.86:0/505007213 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0000cd30 con 0x7efe08107db0 2024-03-22T19:14:17.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.093+0000 7efe05600700 1 -- 172.21.15.86:0/505007213 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00018680 con 0x7efe08107db0 2024-03-22T19:14:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.093+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe081a9740 con 0x7efe08107db0 2024-03-22T19:14:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.093+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe081a9c00 con 0x7efe08107db0 2024-03-22T19:14:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.108+0000 7efe05600700 1 -- 172.21.15.86:0/505007213 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7efe00010430 con 0x7efe08107db0 2024-03-22T19:14:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.112+0000 7efe05600700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efdec0a0b60 0x7efdec0a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.112+0000 7efe05600700 1 -- 172.21.15.86:0/505007213 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7efe00014070 con 0x7efe08107db0 2024-03-22T19:14:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.114+0000 7efe07e00700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efdec0a0b60 0x7efdec0a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.116+0000 7efe07e00700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efdec0a0b60 0x7efdec0a3020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000c00 tx=0x7efdf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:17.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.119+0000 7efe23423b80 1 Processor -- start 2024-03-22T19:14:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.121+0000 7efe23423b80 1 -- start start 2024-03-22T19:14:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.121+0000 7efe23423b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea2258de0 0x556ea2259220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe23423b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea225a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe23423b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a9e0 0x556ea2261270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe23423b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea21b1990 con 0x556ea2258de0 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe23423b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea21af350 con 0x556ea225a020 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe23423b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea21aac40 con 0x556ea225a9e0 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe07e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea225a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe07e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea225a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48824/0 (socket says 172.21.15.86:48824) 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe07e00700 1 -- 172.21.15.86:0/3320822858 learned_addr learned my addr 172.21.15.86:0/3320822858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.122+0000 7efe0d000700 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea2258de0 0x556ea2259220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.123+0000 7efe0d000700 1 -- 172.21.15.86:0/3320822858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a9e0 msgr2=0x556ea2261270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.123+0000 7efe0d000700 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a9e0 0x556ea2261270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.123+0000 7efe0d000700 1 -- 172.21.15.86:0/3320822858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 msgr2=0x556ea225a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.123+0000 7efe0d000700 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea225a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.123+0000 7efe0d000700 1 -- 172.21.15.86:0/3320822858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea202c3b0 con 0x556ea2258de0 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.123+0000 7efe0d000700 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea2258de0 0x556ea2259220 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efe000ca020 tx=0x7efe00099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb6a1a161831b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.124+0000 7efdfe000700 1 -- 172.21.15.86:0/3320822858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0009a260 con 0x556ea2258de0 2024-03-22T19:14:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.124+0000 7efdfe000700 1 -- 172.21.15.86:0/3320822858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe00099b60 con 0x556ea2258de0 2024-03-22T19:14:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.124+0000 7efdfe000700 1 -- 172.21.15.86:0/3320822858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00099050 con 0x556ea2258de0 2024-03-22T19:14:17.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.127+0000 7efe23423b80 1 -- 172.21.15.86:0/3320822858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea2258de0 msgr2=0x556ea2259220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.127+0000 7efe23423b80 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea2258de0 0x556ea2259220 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efe000ca020 tx=0x7efe00099a50 comp rx=0 tx=0).stop 2024-03-22T19:14:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 -- 172.21.15.86:0/3320822858 shutdown_connections 2024-03-22T19:14:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a9e0 0x556ea2261270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea2258de0 0x556ea2259220 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 --2- 172.21.15.86:0/3320822858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea225a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 -- 172.21.15.86:0/3320822858 >> 172.21.15.86:0/3320822858 conn(0x556ea21ce220 msgr2=0x556ea2253e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 -- 172.21.15.86:0/3320822858 shutdown_connections 2024-03-22T19:14:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.128+0000 7efe23423b80 1 -- 172.21.15.86:0/3320822858 wait complete. 2024-03-22T19:14:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.129+0000 7efe23423b80 1 -- start start 2024-03-22T19:14:17.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.130+0000 7efe23423b80 1 Processor -- start 2024-03-22T19:14:17.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.131+0000 7efe23423b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea2262d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.131+0000 7efe23423b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea2263260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.132+0000 7efe23423b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea226a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.132+0000 7efe23423b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea2146d80 con 0x556ea225a9e0 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.132+0000 7efe23423b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea2146a20 con 0x556ea225a020 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.132+0000 7efe23423b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea202e6a0 con 0x556ea2258de0 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.132+0000 7efe23423b80 10 client.0 _ll_register_callbacks cb 0x556ea1fb23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.134+0000 7efe0d000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea2262d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.134+0000 7efe07e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea2263260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.134+0000 7efe07e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea2263260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48826/0 (socket says 172.21.15.86:48826) 2024-03-22T19:14:17.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.134+0000 7efe07e00700 1 -- 172.21.15.86:0/2657814339 learned_addr learned my addr 172.21.15.86:0/2657814339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:17.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.134+0000 7efe0da00700 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea226a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.135+0000 7efe0da00700 1 -- 172.21.15.86:0/2657814339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 msgr2=0x556ea2262d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.135+0000 7efe0da00700 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea2262d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.135+0000 7efe0da00700 1 -- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 msgr2=0x556ea2263260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.135+0000 7efe0da00700 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea2263260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.135+0000 7efe0da00700 1 -- 172.21.15.86:0/2657814339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe081a9c00 con 0x556ea225a9e0 2024-03-22T19:14:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.136+0000 7efe0da00700 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea226a2a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c500 tx=0x7efdf800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d223c0d10a0b45a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.136+0000 7efdf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:14:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.136+0000 7efdf2a00700 1 -- 172.21.15.86:0/2657814339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8011840 con 0x556ea225a9e0 2024-03-22T19:14:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.136+0000 7efdf2a00700 1 -- 172.21.15.86:0/2657814339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7efdf800d040 con 0x556ea225a9e0 2024-03-22T19:14:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.136+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ea224bc30 con 0x556ea225a9e0 2024-03-22T19:14:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.137+0000 7efdf2a00700 1 -- 172.21.15.86:0/2657814339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf80119a0 con 0x556ea225a9e0 2024-03-22T19:14:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.137+0000 7efe23423b80 10 client.15987 fetch_fsmap learned FSMap version 33 2024-03-22T19:14:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.137+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556ea1fd8b60 con 0x556ea225a9e0 2024-03-22T19:14:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.138+0000 7efdf2a00700 1 -- 172.21.15.86:0/2657814339 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ea224bc30 con 0x556ea225a9e0 2024-03-22T19:14:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.138+0000 7efdf2a00700 1 -- 172.21.15.86:0/2657814339 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7490+0+0 (secure 0 0 0) 0x7efdf8010b60 con 0x556ea225a9e0 2024-03-22T19:14:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.138+0000 7efe23423b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 33 2024-03-22T19:14:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.138+0000 7efe23423b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:14:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.138+0000 7efe23423b80 20 client.15987 populate_metadata read hostname 'smithi086' 2024-03-22T19:14:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.139+0000 7efe23423b80 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea226ec80 0x556ea22710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.139+0000 7efe23423b80 4 client.15987 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:14:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.139+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ea21a3700 con 0x556ea226ec80 2024-03-22T19:14:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.140+0000 7efe0d000700 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea226ec80 0x556ea22710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.145+0000 7efe0d000700 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea226ec80 0x556ea22710a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3986da8ea38ab5 server_cookie=dc3b0a99df8150e8 in_seq=0 out_seq=0 2024-03-22T19:14:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.145+0000 7efdf2a00700 10 client.15987 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.146+0000 7efdf2a00700 1 -- 172.21.15.86:0/2657814339 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556ea21a3700 con 0x556ea226ec80 2024-03-22T19:14:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.146+0000 7efdf2a00700 10 client.15987 handle_command_reply: tid=0 2024-03-22T19:14:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.146+0000 7efe23423b80 1 client.15987 shutdown 2024-03-22T19:14:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.147+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 msgr2=0x556ea226a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.147+0000 7efe23423b80 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea226a2a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c500 tx=0x7efdf800c810 comp rx=0 tx=0).stop 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 shutdown_connections 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efe23423b80 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea226ec80 0x556ea22710a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efe23423b80 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea226a2a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efe23423b80 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea2262d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efe23423b80 1 --2- 172.21.15.86:0/2657814339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a020 0x556ea2263260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 >> 172.21.15.86:0/2657814339 conn(0x556ea21ce220 msgr2=0x556ea2253350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.148+0000 7efdf2a00700 0 client.15987 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.149+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 shutdown_connections 2024-03-22T19:14:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.149+0000 7efe23423b80 1 -- 172.21.15.86:0/2657814339 wait complete. 2024-03-22T19:14:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.150+0000 7efe23423b80 20 client.15987 trim_cache size 0 max 16384 2024-03-22T19:14:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.185+0000 7efe23423b80 1 Processor -- start 2024-03-22T19:14:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.186+0000 7efe23423b80 1 -- start start 2024-03-22T19:14:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.188+0000 7efe23423b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea2258de0 0x556ea22681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe23423b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a020 0x556ea225a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe0d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea2258de0 0x556ea22681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe0d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea2258de0 0x556ea22681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:48838/0 (socket says 172.21.15.86:48838) 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe23423b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea2262230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe23423b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea224e400 con 0x556ea225a9e0 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe23423b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea2262890 con 0x556ea2258de0 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe23423b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea22595c0 con 0x556ea225a020 2024-03-22T19:14:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.189+0000 7efe0d000700 1 -- 172.21.15.86:0/3528349409 learned_addr learned my addr 172.21.15.86:0/3528349409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.190+0000 7efe07e00700 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a020 0x556ea225a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.190+0000 7efe0da00700 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea2262230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.192+0000 7efe07e00700 1 -- 172.21.15.86:0/3528349409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea2258de0 msgr2=0x556ea22681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.192+0000 7efe07e00700 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea2258de0 0x556ea22681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.192+0000 7efe07e00700 1 -- 172.21.15.86:0/3528349409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 msgr2=0x556ea2262230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.192+0000 7efe07e00700 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea2262230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.192+0000 7efe07e00700 1 -- 172.21.15.86:0/3528349409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea2170ea0 con 0x556ea225a020 2024-03-22T19:14:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.196+0000 7efe07e00700 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a020 0x556ea225a400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efdf40074e0 tx=0x7efdf401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb68f31860bfa29a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.199+0000 7efdfea00700 1 -- 172.21.15.86:0/3528349409 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400f040 con 0x556ea225a020 2024-03-22T19:14:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.199+0000 7efdfea00700 1 -- 172.21.15.86:0/3528349409 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf40056a0 con 0x556ea225a020 2024-03-22T19:14:17.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.199+0000 7efdfea00700 1 -- 172.21.15.86:0/3528349409 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400cc10 con 0x556ea225a020 2024-03-22T19:14:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.200+0000 7efe23423b80 1 -- 172.21.15.86:0/3528349409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a020 msgr2=0x556ea225a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.200+0000 7efe23423b80 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a020 0x556ea225a400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efdf40074e0 tx=0x7efdf401a000 comp rx=0 tx=0).stop 2024-03-22T19:14:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.201+0000 7efe23423b80 1 -- 172.21.15.86:0/3528349409 shutdown_connections 2024-03-22T19:14:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.201+0000 7efe23423b80 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a9e0 0x556ea2262230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.201+0000 7efe23423b80 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea2258de0 0x556ea22681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.201+0000 7efe23423b80 1 --2- 172.21.15.86:0/3528349409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea225a020 0x556ea225a400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.201+0000 7efe23423b80 1 -- 172.21.15.86:0/3528349409 >> 172.21.15.86:0/3528349409 conn(0x556ea21ce220 msgr2=0x556ea2270c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.202+0000 7efe23423b80 1 -- 172.21.15.86:0/3528349409 shutdown_connections 2024-03-22T19:14:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.202+0000 7efe23423b80 1 -- 172.21.15.86:0/3528349409 wait complete. 2024-03-22T19:14:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.203+0000 7efe23423b80 1 -- start start 2024-03-22T19:14:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.204+0000 7efe23423b80 1 Processor -- start 2024-03-22T19:14:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.205+0000 7efe23423b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea22c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe23423b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 0x556ea22c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe23423b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a9e0 0x556ea22c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe23423b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea22c8100 con 0x556ea225a020 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe23423b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea22c8270 con 0x556ea225a9e0 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe23423b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea22c8430 con 0x556ea2258de0 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe23423b80 10 client.0 _ll_register_callbacks cb 0x7efdf8007b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe0d000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea22c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.206+0000 7efe07e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 0x556ea22c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.207+0000 7efe07e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 0x556ea22c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42070/0 (socket says 172.21.15.86:42070) 2024-03-22T19:14:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.207+0000 7efe07e00700 1 -- 172.21.15.86:0/3586341159 learned_addr learned my addr 172.21.15.86:0/3586341159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.207+0000 7efe0da00700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a9e0 0x556ea22c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.208+0000 7efe07e00700 1 -- 172.21.15.86:0/3586341159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 msgr2=0x556ea22c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.208+0000 7efe07e00700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea22c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.209+0000 7efe07e00700 1 -- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a9e0 msgr2=0x556ea22c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.209+0000 7efe07e00700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a9e0 0x556ea22c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.209+0000 7efe07e00700 1 -- 172.21.15.86:0/3586341159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe08110330 con 0x556ea225a020 2024-03-22T19:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.209+0000 7efe0d000700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea22c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.210+0000 7efe07e00700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 0x556ea22c54c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efdf4003710 tx=0x7efdf4003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2fa67dbdf9fe8e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.210+0000 7efdf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.210+0000 7efdf2a00700 1 -- 172.21.15.86:0/3586341159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400f040 con 0x556ea225a020 2024-03-22T19:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.211+0000 7efdf2a00700 1 -- 172.21.15.86:0/3586341159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7efdf4038070 con 0x556ea225a020 2024-03-22T19:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.211+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe00099b60 con 0x556ea225a020 2024-03-22T19:14:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.215+0000 7efdf2a00700 1 -- 172.21.15.86:0/3586341159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf4003b70 con 0x556ea225a020 2024-03-22T19:14:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.215+0000 7efe23423b80 10 client.15993 fetch_fsmap learned FSMap version 33 2024-03-22T19:14:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.215+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556ea2030cd0 con 0x556ea225a020 2024-03-22T19:14:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.217+0000 7efdf2a00700 1 -- 172.21.15.86:0/3586341159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe00099b60 con 0x556ea225a020 2024-03-22T19:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efdf2a00700 1 -- 172.21.15.86:0/3586341159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7490+0+0 (secure 0 0 0) 0x7efdf4004a60 con 0x556ea225a020 2024-03-22T19:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efe23423b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 33 2024-03-22T19:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efe23423b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efe23423b80 20 client.15993 populate_metadata read hostname 'smithi086' 2024-03-22T19:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efe23423b80 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea22c9b00 0x556ea22cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efe23423b80 4 client.15993 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.218+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ea1ff9cf0 con 0x556ea22c9b00 2024-03-22T19:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.220+0000 7efe0d000700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea22c9b00 0x556ea22cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.221+0000 7efe0d000700 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea22c9b00 0x556ea22cbf20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e507e16f407e26 server_cookie=b7402287ca385c79 in_seq=0 out_seq=0 2024-03-22T19:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.221+0000 7efdf2a00700 10 client.15993 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.221+0000 7efdf2a00700 1 -- 172.21.15.86:0/3586341159 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556ea1ff9cf0 con 0x556ea22c9b00 2024-03-22T19:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.221+0000 7efdf2a00700 10 client.15993 handle_command_reply: tid=0 2024-03-22T19:14:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.222+0000 7efe23423b80 1 client.15993 shutdown 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 msgr2=0x556ea22c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 0x556ea22c54c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efdf4003710 tx=0x7efdf4003740 comp rx=0 tx=0).stop 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 shutdown_connections 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x556ea22c9b00 0x556ea22cbf20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ea225a9e0 0x556ea22c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ea2258de0 0x556ea22c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 --2- 172.21.15.86:0/3586341159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ea225a020 0x556ea22c54c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.223+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 >> 172.21.15.86:0/3586341159 conn(0x556ea21ce220 msgr2=0x556ea225da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.224+0000 7efdf2a00700 0 client.15993 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.224+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 shutdown_connections 2024-03-22T19:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.225+0000 7efe23423b80 1 -- 172.21.15.86:0/3586341159 wait complete. 2024-03-22T19:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.225+0000 7efe23423b80 20 client.15993 trim_cache size 0 max 16384 2024-03-22T19:14:17.226 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (5 inodes in the stack)", 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: "e34a3ccd-deee-4f94-9659-5669e603b7be": { 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e34a3ccd-deee-4f94-9659-5669e603b7be", 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:14:17.227 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:14:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efdec0a0b60 msgr2=0x7efdec0a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efdec0a0b60 0x7efdec0a3020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000c00 tx=0x7efdf4005fb0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 msgr2=0x7efe081a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:17.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 0x7efe081a16b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe0000e9a0 tx=0x7efe0000ed60 comp rx=0 tx=0).stop 2024-03-22T19:14:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 shutdown_connections 2024-03-22T19:14:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7efdec0a0b60 0x7efdec0a3020 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0810d250 0x7efe081a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe08107db0 0x7efe081a16b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 --2- 172.21.15.86:0/505007213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0810a770 0x7efe081a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.230+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 >> 172.21.15.86:0/505007213 conn(0x7efe0806da20 msgr2=0x7efe08108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:17.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.232+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 shutdown_connections 2024-03-22T19:14:17.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:17.232+0000 7efe0e400700 1 -- 172.21.15.86:0/505007213 wait complete. 2024-03-22T19:14:17.276 INFO:tasks.cephfs.filesystem:scrub status for tag:e34a3ccd-deee-4f94-9659-5669e603b7be - {'path': '/', 'tag': 'e34a3ccd-deee-4f94-9659-5669e603b7be', 'options': 'recursive,force'} 2024-03-22T19:14:17.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:17 smithi131 ceph-mon[78559]: pgmap v385: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 153 MiB/s wr, 19.97k op/s 2024-03-22T19:14:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:17 smithi086 ceph-mon[38032]: pgmap v385: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 153 MiB/s wr, 19.97k op/s 2024-03-22T19:14:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:17 smithi167 ceph-mon[47728]: pgmap v385: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 153 MiB/s wr, 19.97k op/s 2024-03-22T19:14:19.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:19 smithi131 ceph-mon[78559]: pgmap v386: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 MiB/s wr, 18.97k op/s 2024-03-22T19:14:19.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:19 smithi086 ceph-mon[38032]: pgmap v386: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 MiB/s wr, 18.97k op/s 2024-03-22T19:14:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:19 smithi167 ceph-mon[47728]: pgmap v386: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 MiB/s wr, 18.97k op/s 2024-03-22T19:14:20.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:20 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:14:20.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:20 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:14:20.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:20 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:14:21.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:21 smithi131 ceph-mon[78559]: pgmap v387: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 136 MiB/s wr, 18.67k op/s 2024-03-22T19:14:21.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:14:21.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:21 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:21.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:21 smithi086 ceph-mon[38032]: pgmap v387: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 136 MiB/s wr, 18.67k op/s 2024-03-22T19:14:21.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:14:21.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:21 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:21 smithi167 ceph-mon[47728]: pgmap v387: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 136 MiB/s wr, 18.67k op/s 2024-03-22T19:14:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:14:21.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:21 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:22 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:22 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:22 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:22 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:22 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:22 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:22 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:22 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:22 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:22 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:22 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:22 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:22 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:22 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:22.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:22 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:23.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:23 smithi131 ceph-mon[78559]: pgmap v388: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 140 MiB/s wr, 20.46k op/s 2024-03-22T19:14:23.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:14:23.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:14:23.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:23 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:23 smithi086 ceph-mon[38032]: pgmap v388: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 140 MiB/s wr, 20.46k op/s 2024-03-22T19:14:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:14:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:14:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:23 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:23.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:23 smithi167 ceph-mon[47728]: pgmap v388: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 140 MiB/s wr, 20.46k op/s 2024-03-22T19:14:23.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:14:23.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:14:23.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:23 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:14:25.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:25 smithi086 ceph-mon[38032]: pgmap v389: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 109 MiB/s wr, 16.45k op/s 2024-03-22T19:14:25.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:25 smithi167 ceph-mon[47728]: pgmap v389: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 109 MiB/s wr, 16.45k op/s 2024-03-22T19:14:25.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:25 smithi131 ceph-mon[78559]: pgmap v389: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 109 MiB/s wr, 16.45k op/s 2024-03-22T19:14:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:27 smithi167 ceph-mon[47728]: pgmap v390: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 118 MiB/s wr, 17.96k op/s 2024-03-22T19:14:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:27 smithi131 ceph-mon[78559]: pgmap v390: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 118 MiB/s wr, 17.96k op/s 2024-03-22T19:14:27.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:27 smithi086 ceph-mon[38032]: pgmap v390: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 118 MiB/s wr, 17.96k op/s 2024-03-22T19:14:29.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:29 smithi131 ceph-mon[78559]: pgmap v391: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 113 MiB/s wr, 17.61k op/s 2024-03-22T19:14:29.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:29 smithi086 ceph-mon[38032]: pgmap v391: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 113 MiB/s wr, 17.61k op/s 2024-03-22T19:14:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:29 smithi167 ceph-mon[47728]: pgmap v391: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 113 MiB/s wr, 17.61k op/s 2024-03-22T19:14:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:31 smithi131 ceph-mon[78559]: pgmap v392: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 104 MiB/s wr, 16.33k op/s 2024-03-22T19:14:31.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:31 smithi086 ceph-mon[38032]: pgmap v392: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 104 MiB/s wr, 16.33k op/s 2024-03-22T19:14:31.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:31 smithi167 ceph-mon[47728]: pgmap v392: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 104 MiB/s wr, 16.33k op/s 2024-03-22T19:14:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:33 smithi131 ceph-mon[78559]: pgmap v393: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 118 MiB/s wr, 18.36k op/s 2024-03-22T19:14:33.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:33 smithi086 ceph-mon[38032]: pgmap v393: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 118 MiB/s wr, 18.36k op/s 2024-03-22T19:14:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:33 smithi167 ceph-mon[47728]: pgmap v393: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 118 MiB/s wr, 18.36k op/s 2024-03-22T19:14:35.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:35 smithi131 ceph-mon[78559]: pgmap v394: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 93 MiB/s wr, 14.26k op/s 2024-03-22T19:14:35.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:35 smithi086 ceph-mon[38032]: pgmap v394: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 93 MiB/s wr, 14.26k op/s 2024-03-22T19:14:35.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:35 smithi167 ceph-mon[47728]: pgmap v394: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 93 MiB/s wr, 14.26k op/s 2024-03-22T19:14:37.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:37 smithi167 ceph-mon[47728]: pgmap v395: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 104 MiB/s wr, 15.86k op/s 2024-03-22T19:14:37.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:37 smithi131 ceph-mon[78559]: pgmap v395: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 104 MiB/s wr, 15.86k op/s 2024-03-22T19:14:37.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:37 smithi086 ceph-mon[38032]: pgmap v395: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 104 MiB/s wr, 15.86k op/s 2024-03-22T19:14:39.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:39 smithi131 ceph-mon[78559]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 117 MiB/s wr, 18.01k op/s 2024-03-22T19:14:39.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:39 smithi086 ceph-mon[38032]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 117 MiB/s wr, 18.01k op/s 2024-03-22T19:14:39.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:39 smithi167 ceph-mon[47728]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 117 MiB/s wr, 18.01k op/s 2024-03-22T19:14:41.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:41 smithi131 ceph-mon[78559]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 98 MiB/s wr, 15.03k op/s 2024-03-22T19:14:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:41 smithi086 ceph-mon[38032]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 98 MiB/s wr, 15.03k op/s 2024-03-22T19:14:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:41 smithi167 ceph-mon[47728]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 98 MiB/s wr, 15.03k op/s 2024-03-22T19:14:43.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:43 smithi131 ceph-mon[78559]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 115 MiB/s wr, 17.40k op/s 2024-03-22T19:14:43.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:43 smithi086 ceph-mon[38032]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 115 MiB/s wr, 17.40k op/s 2024-03-22T19:14:43.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:43 smithi167 ceph-mon[47728]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 115 MiB/s wr, 17.40k op/s 2024-03-22T19:14:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:45 smithi131 ceph-mon[78559]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 100 MiB/s wr, 14.92k op/s 2024-03-22T19:14:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:45 smithi086 ceph-mon[38032]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 100 MiB/s wr, 14.92k op/s 2024-03-22T19:14:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:45 smithi167 ceph-mon[47728]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 100 MiB/s wr, 14.92k op/s 2024-03-22T19:14:47.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:14:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:47 smithi167 ceph-mon[47728]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 113 MiB/s wr, 16.86k op/s 2024-03-22T19:14:47.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:47 smithi131 ceph-mon[78559]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 113 MiB/s wr, 16.86k op/s 2024-03-22T19:14:47.887 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:47 smithi086 ceph-mon[38032]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 113 MiB/s wr, 16.86k op/s 2024-03-22T19:14:49.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:49 smithi131 ceph-mon[78559]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 118 MiB/s wr, 17.85k op/s 2024-03-22T19:14:49.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:49 smithi167 ceph-mon[47728]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 118 MiB/s wr, 17.85k op/s 2024-03-22T19:14:50.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:49 smithi086 ceph-mon[38032]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 118 MiB/s wr, 17.85k op/s 2024-03-22T19:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 -- 172.21.15.86:0/3400044865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340ff600 msgr2=0x7fe9340ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 --2- 172.21.15.86:0/3400044865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340ff600 0x7fe9340ffa20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe9240099c0 tx=0x7fe924009cd0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 -- 172.21.15.86:0/3400044865 shutdown_connections 2024-03-22T19:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 --2- 172.21.15.86:0/3400044865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934100920 0x7fe934100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 --2- 172.21.15.86:0/3400044865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340ff600 0x7fe9340ffa20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 --2- 172.21.15.86:0/3400044865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340fff60 0x7fe9341003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.512+0000 7fe93ac00700 1 -- 172.21.15.86:0/3400044865 >> 172.21.15.86:0/3400044865 conn(0x7fe9340fb1a0 msgr2=0x7fe9340fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.513+0000 7fe93ac00700 1 -- 172.21.15.86:0/3400044865 shutdown_connections 2024-03-22T19:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.513+0000 7fe93ac00700 1 -- 172.21.15.86:0/3400044865 wait complete. 2024-03-22T19:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 Processor -- start 2024-03-22T19:14:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 -- start start 2024-03-22T19:14:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340ff600 0x7fe934198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340fff60 0x7fe9341993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934100920 0x7fe9341a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341a09a0 con 0x7fe9340fff60 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341a0ae0 con 0x7fe934100920 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.514+0000 7fe93ac00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341a0c70 con 0x7fe9340ff600 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340ff600 0x7fe934198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340ff600 0x7fe934198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:46018/0 (socket says 172.21.15.86:46018) 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340fff60 0x7fe9341993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933e00700 1 -- 172.21.15.86:0/2025167786 learned_addr learned my addr 172.21.15.86:0/2025167786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe938c00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934100920 0x7fe9341a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933400700 1 -- 172.21.15.86:0/2025167786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340ff600 msgr2=0x7fe934198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933400700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340ff600 0x7fe934198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933400700 1 -- 172.21.15.86:0/2025167786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934100920 msgr2=0x7fe9341a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933400700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934100920 0x7fe9341a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.515+0000 7fe933400700 1 -- 172.21.15.86:0/2025167786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240095f0 con 0x7fe9340fff60 2024-03-22T19:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.517+0000 7fe933400700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340fff60 0x7fe9341993b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d950 tx=0x7fe91c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.518+0000 7fe930c00700 1 -- 172.21.15.86:0/2025167786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f890 con 0x7fe9340fff60 2024-03-22T19:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.518+0000 7fe930c00700 1 -- 172.21.15.86:0/2025167786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe91c010460 con 0x7fe9340fff60 2024-03-22T19:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.518+0000 7fe930c00700 1 -- 172.21.15.86:0/2025167786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00e610 con 0x7fe9340fff60 2024-03-22T19:14:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.519+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9341019b0 con 0x7fe9340fff60 2024-03-22T19:14:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.519+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe934101e40 con 0x7fe9340fff60 2024-03-22T19:14:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.524+0000 7fe930c00700 1 -- 172.21.15.86:0/2025167786 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe91c0108d0 con 0x7fe9340fff60 2024-03-22T19:14:51.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.526+0000 7fe930c00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe9180a0b90 0x7fe9180a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.526+0000 7fe930c00700 1 -- 172.21.15.86:0/2025167786 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe91c0cbda0 con 0x7fe9340fff60 2024-03-22T19:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.528+0000 7fe933e00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe9180a0b90 0x7fe9180a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.529+0000 7fe933e00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe9180a0b90 0x7fe9180a3050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe924009f50 tx=0x7fe924005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:51.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 Processor -- start 2024-03-22T19:14:51.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 -- start start 2024-03-22T19:14:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff59d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59d580 0x55d1ff59de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59e420 0x55d1ff59e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff4f5990 con 0x55d1ff59d580 2024-03-22T19:14:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff4f3350 con 0x55d1ff59e420 2024-03-22T19:14:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.554+0000 7fe94e783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff4eec40 con 0x55d1ff59cc20 2024-03-22T19:14:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.555+0000 7fe938c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59e420 0x55d1ff59e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.555+0000 7fe933e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff59d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.555+0000 7fe933e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff59d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:46024/0 (socket says 172.21.15.86:46024) 2024-03-22T19:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.555+0000 7fe933e00700 1 -- 172.21.15.86:0/1613328067 learned_addr learned my addr 172.21.15.86:0/1613328067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.556+0000 7fe933400700 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59d580 0x55d1ff59de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe933e00700 1 -- 172.21.15.86:0/1613328067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59e420 msgr2=0x55d1ff59e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe933e00700 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59e420 0x55d1ff59e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe933e00700 1 -- 172.21.15.86:0/1613328067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59d580 msgr2=0x55d1ff59de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe933e00700 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59d580 0x55d1ff59de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe933e00700 1 -- 172.21.15.86:0/1613328067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ff3703b0 con 0x55d1ff59cc20 2024-03-22T19:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe938c00700 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59e420 0x55d1ff59e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.558+0000 7fe933e00700 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff59d040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe924005910 tx=0x7fe924020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df365c5fd6ff4933 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.559+0000 7fe929200700 1 -- 172.21.15.86:0/1613328067 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400be20 con 0x55d1ff59cc20 2024-03-22T19:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.559+0000 7fe929200700 1 -- 172.21.15.86:0/1613328067 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92401e040 con 0x55d1ff59cc20 2024-03-22T19:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.559+0000 7fe929200700 1 -- 172.21.15.86:0/1613328067 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240049c0 con 0x55d1ff59cc20 2024-03-22T19:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.559+0000 7fe94e783b80 1 -- 172.21.15.86:0/1613328067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 msgr2=0x55d1ff59d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.559+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff59d040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe924005910 tx=0x7fe924020000 comp rx=0 tx=0).stop 2024-03-22T19:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 -- 172.21.15.86:0/1613328067 shutdown_connections 2024-03-22T19:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59e420 0x55d1ff59e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff59d040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1613328067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59d580 0x55d1ff59de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 -- 172.21.15.86:0/1613328067 >> 172.21.15.86:0/1613328067 conn(0x55d1ff512220 msgr2=0x55d1ff597bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 -- 172.21.15.86:0/1613328067 shutdown_connections 2024-03-22T19:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.560+0000 7fe94e783b80 1 -- 172.21.15.86:0/1613328067 wait complete. 2024-03-22T19:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.561+0000 7fe94e783b80 1 -- start start 2024-03-22T19:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.561+0000 7fe94e783b80 1 Processor -- start 2024-03-22T19:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59d580 0x55d1ff5a7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff5adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff48ad80 con 0x55d1ff59e420 2024-03-22T19:14:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff48aa20 con 0x55d1ff59cc20 2024-03-22T19:14:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff3726a0 con 0x55d1ff59d580 2024-03-22T19:14:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.562+0000 7fe94e783b80 10 client.0 _ll_register_callbacks cb 0x55d1ff2f63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:35376/0 (socket says 172.21.15.86:35376) 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 -- 172.21.15.86:0/201430841 learned_addr learned my addr 172.21.15.86:0/201430841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933400700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59d580 0x55d1ff5a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 -- 172.21.15.86:0/201430841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59d580 msgr2=0x55d1ff5a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59d580 0x55d1ff5a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 -- 172.21.15.86:0/201430841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 msgr2=0x55d1ff5adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff5adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 -- 172.21.15.86:0/201430841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93410bca0 con 0x55d1ff59cc20 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933400700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59d580 0x55d1ff5a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.563+0000 7fe933e00700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a6ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe924005f50 tx=0x7fe924004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4640929959e4cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe921600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe921600700 1 -- 172.21.15.86:0/201430841 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924003ac0 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe921600700 1 -- 172.21.15.86:0/201430841 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe924003c20 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1ff58fad0 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe921600700 1 -- 172.21.15.86:0/201430841 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe924003de0 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe94e783b80 10 client.26105 fetch_fsmap learned FSMap version 34 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d1ff31cb60 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.564+0000 7fe921600700 1 -- 172.21.15.86:0/201430841 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1ff58fad0 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe921600700 1 -- 172.21.15.86:0/201430841 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe924044070 con 0x55d1ff59cc20 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe94e783b80 10 client.26105 fetch_fsmap finished waiting for FSMap version 34 2024-03-22T19:14:51.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe94e783b80 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe94e783b80 20 client.26105 populate_metadata read hostname 'smithi086' 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe94e783b80 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff5b24c0 0x55d1ff5b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe94e783b80 4 client.26105 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1ff4e7700 con 0x55d1ff5b24c0 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.565+0000 7fe933400700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff5b24c0 0x55d1ff5b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.566+0000 7fe921600700 10 client.26105 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.566+0000 7fe933400700 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff5b24c0 0x55d1ff5b48e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733a21a90e08f44a server_cookie=291e67baf1df5a88 in_seq=0 out_seq=0 2024-03-22T19:14:51.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.566+0000 7fe921600700 1 -- 172.21.15.86:0/201430841 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d1ff4e7700 con 0x55d1ff5b24c0 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.566+0000 7fe921600700 10 client.26105 handle_command_reply: tid=0 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.566+0000 7fe94e783b80 1 client.26105 shutdown 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 msgr2=0x55d1ff5a6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a6ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe924005f50 tx=0x7fe924004970 comp rx=0 tx=0).stop 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 shutdown_connections 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff5b24c0 0x55d1ff5b48e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe921600700 0 client.26105 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff5adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a6ac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 --2- 172.21.15.86:0/201430841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59d580 0x55d1ff5a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.567+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 >> 172.21.15.86:0/201430841 conn(0x55d1ff512220 msgr2=0x55d1ff5971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:51.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.568+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 shutdown_connections 2024-03-22T19:14:51.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.569+0000 7fe94e783b80 1 -- 172.21.15.86:0/201430841 wait complete. 2024-03-22T19:14:51.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.569+0000 7fe94e783b80 20 client.26105 trim_cache size 0 max 16384 2024-03-22T19:14:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.598+0000 7fe94e783b80 1 Processor -- start 2024-03-22T19:14:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.598+0000 7fe94e783b80 1 -- start start 2024-03-22T19:14:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.598+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.598+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff59e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe94e783b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff591f80 con 0x55d1ff59e420 2024-03-22T19:14:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe94e783b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff5b9600 con 0x55d1ff59d580 2024-03-22T19:14:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe94e783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff5a5440 con 0x55d1ff59cc20 2024-03-22T19:14:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe933400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe938c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff59e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe933400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:35386/0 (socket says 172.21.15.86:35386) 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe933400700 1 -- 172.21.15.86:0/1580107526 learned_addr learned my addr 172.21.15.86:0/1580107526 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.599+0000 7fe933e00700 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe933400700 1 -- 172.21.15.86:0/1580107526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 msgr2=0x55d1ff5a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe933400700 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe933400700 1 -- 172.21.15.86:0/1580107526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 msgr2=0x55d1ff59e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe933400700 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff59e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe933400700 1 -- 172.21.15.86:0/1580107526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ff4b4ea0 con 0x55d1ff59d580 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe933400700 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5abf50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe91c010fd0 tx=0x7fe91c0c9760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b563c0327a49836 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe929c00700 1 -- 172.21.15.86:0/1580107526 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c09d250 con 0x55d1ff59d580 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe929c00700 1 -- 172.21.15.86:0/1580107526 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe91c0a18a0 con 0x55d1ff59d580 2024-03-22T19:14:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.600+0000 7fe929c00700 1 -- 172.21.15.86:0/1580107526 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0998e0 con 0x55d1ff59d580 2024-03-22T19:14:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 -- 172.21.15.86:0/1580107526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 msgr2=0x55d1ff5abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5abf50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe91c010fd0 tx=0x7fe91c0c9760 comp rx=0 tx=0).stop 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 -- 172.21.15.86:0/1580107526 shutdown_connections 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59e420 0x55d1ff59e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 --2- 172.21.15.86:0/1580107526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5abf50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 -- 172.21.15.86:0/1580107526 >> 172.21.15.86:0/1580107526 conn(0x55d1ff512220 msgr2=0x55d1ff593a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 -- 172.21.15.86:0/1580107526 shutdown_connections 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.601+0000 7fe94e783b80 1 -- 172.21.15.86:0/1580107526 wait complete. 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.602+0000 7fe94e783b80 1 -- start start 2024-03-22T19:14:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.602+0000 7fe94e783b80 1 Processor -- start 2024-03-22T19:14:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.602+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe94e783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 0x55d1ff5aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe94e783b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff5ab3a0 con 0x55d1ff59cc20 2024-03-22T19:14:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe94e783b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff5ab510 con 0x55d1ff59d580 2024-03-22T19:14:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe94e783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ff5ab6d0 con 0x55d1ff59e420 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe94e783b80 10 client.0 _ll_register_callbacks cb 0x7fe9240053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe933e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe938c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 0x55d1ff5aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe938c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 0x55d1ff5aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:46046/0 (socket says 172.21.15.86:46046) 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe938c00700 1 -- 172.21.15.86:0/4191774297 learned_addr learned my addr 172.21.15.86:0/4191774297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.603+0000 7fe933400700 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe938c00700 1 -- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 msgr2=0x55d1ff5a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe938c00700 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe938c00700 1 -- 172.21.15.86:0/4191774297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59cc20 msgr2=0x55d1ff5a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe938c00700 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe938c00700 1 -- 172.21.15.86:0/4191774297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe934101e40 con 0x55d1ff59e420 2024-03-22T19:14:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe938c00700 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 0x55d1ff5aae60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe92c007af0 tx=0x7fe92c007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f413c4ca2b6e0ce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:14:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe921600700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:14:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe921600700 1 -- 172.21.15.86:0/4191774297 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0269c0 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe921600700 1 -- 172.21.15.86:0/4191774297 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe92c026b20 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.604+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe92401e040 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.605+0000 7fe921600700 1 -- 172.21.15.86:0/4191774297 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c026ce0 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.605+0000 7fe94e783b80 10 client.26107 fetch_fsmap learned FSMap version 34 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.605+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d1ff374cd0 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.605+0000 7fe921600700 1 -- 172.21.15.86:0/4191774297 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92401e040 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe921600700 1 -- 172.21.15.86:0/4191774297 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe92c028420 con 0x55d1ff59e420 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe94e783b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 34 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe94e783b80 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:14:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe94e783b80 20 client.26107 populate_metadata read hostname 'smithi086' 2024-03-22T19:14:51.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe94e783b80 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff604cb0 0x55d1ff6070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:14:51.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe94e783b80 4 client.26107 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:14:51.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1ff33dcf0 con 0x55d1ff604cb0 2024-03-22T19:14:51.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.606+0000 7fe933e00700 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff604cb0 0x55d1ff6070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:14:51.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.607+0000 7fe933e00700 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff604cb0 0x55d1ff6070d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a482b2886f49d01 server_cookie=4eee0d25a18b54e6 in_seq=0 out_seq=0 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.607+0000 7fe921600700 10 client.26107 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.607+0000 7fe921600700 1 -- 172.21.15.86:0/4191774297 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d1ff33dcf0 con 0x55d1ff604cb0 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.607+0000 7fe921600700 10 client.26107 handle_command_reply: tid=0 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.607+0000 7fe94e783b80 1 client.26107 shutdown 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 msgr2=0x55d1ff5aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 0x55d1ff5aae60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe92c007af0 tx=0x7fe92c007e00 comp rx=0 tx=0).stop 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 shutdown_connections 2024-03-22T19:14:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d1ff604cb0 0x55d1ff6070d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1ff59e420 0x55d1ff5aae60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1ff59cc20 0x55d1ff5a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 --2- 172.21.15.86:0/4191774297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ff59d580 0x55d1ff5a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 >> 172.21.15.86:0/4191774297 conn(0x55d1ff512220 msgr2=0x55d1ff5a5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe921600700 0 client.26107 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 shutdown_connections 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 1 -- 172.21.15.86:0/4191774297 wait complete. 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.608+0000 7fe94e783b80 20 client.26107 trim_cache size 0 max 16384 2024-03-22T19:14:51.611 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: "e34a3ccd-deee-4f94-9659-5669e603b7be": { 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e34a3ccd-deee-4f94-9659-5669e603b7be", 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:14:51.612 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe9180a0b90 msgr2=0x7fe9180a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe9180a0b90 0x7fe9180a3050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe924009f50 tx=0x7fe924005fb0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340fff60 msgr2=0x7fe9341993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340fff60 0x7fe9341993b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d950 tx=0x7fe91c00dc60 comp rx=0 tx=0).stop 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 shutdown_connections 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe9180a0b90 0x7fe9180a3050 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934100920 0x7fe9341a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9340ff600 0x7fe934198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 --2- 172.21.15.86:0/2025167786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9340fff60 0x7fe9341993b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.609+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 >> 172.21.15.86:0/2025167786 conn(0x7fe9340fb1a0 msgr2=0x7fe9340fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.610+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 shutdown_connections 2024-03-22T19:14:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:14:51.610+0000 7fe93ac00700 1 -- 172.21.15.86:0/2025167786 wait complete. 2024-03-22T19:14:51.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:51 smithi086 ceph-mon[38032]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 96 MiB/s wr, 14.33k op/s 2024-03-22T19:14:51.642 INFO:tasks.cephfs.filesystem:scrub status for tag:e34a3ccd-deee-4f94-9659-5669e603b7be - {'path': '/', 'tag': 'e34a3ccd-deee-4f94-9659-5669e603b7be', 'options': 'recursive,force'} 2024-03-22T19:14:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:51 smithi131 ceph-mon[78559]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 96 MiB/s wr, 14.33k op/s 2024-03-22T19:14:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:51 smithi167 ceph-mon[47728]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 96 MiB/s wr, 14.33k op/s 2024-03-22T19:14:53.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:53 smithi131 ceph-mon[78559]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 108 MiB/s wr, 16.22k op/s 2024-03-22T19:14:53.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:53 smithi086 ceph-mon[38032]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 108 MiB/s wr, 16.22k op/s 2024-03-22T19:14:53.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:53 smithi167 ceph-mon[47728]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 108 MiB/s wr, 16.22k op/s 2024-03-22T19:14:55.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:55 smithi131 ceph-mon[78559]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 107 MiB/s wr, 16.13k op/s 2024-03-22T19:14:55.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:55 smithi086 ceph-mon[38032]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 107 MiB/s wr, 16.13k op/s 2024-03-22T19:14:55.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:55 smithi167 ceph-mon[47728]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 107 MiB/s wr, 16.13k op/s 2024-03-22T19:14:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:57 smithi167 ceph-mon[47728]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 107 MiB/s wr, 16.29k op/s 2024-03-22T19:14:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:57 smithi131 ceph-mon[78559]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 107 MiB/s wr, 16.29k op/s 2024-03-22T19:14:57.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:57 smithi086 ceph-mon[38032]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 107 MiB/s wr, 16.29k op/s 2024-03-22T19:14:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:14:59 smithi167 ceph-mon[47728]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 107 MiB/s wr, 16.71k op/s 2024-03-22T19:14:59.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:14:59 smithi131 ceph-mon[78559]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 107 MiB/s wr, 16.71k op/s 2024-03-22T19:14:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:14:59 smithi086 ceph-mon[38032]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 107 MiB/s wr, 16.71k op/s 2024-03-22T19:15:01.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:01 smithi131 ceph-mon[78559]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 93 MiB/s wr, 14.26k op/s 2024-03-22T19:15:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:01 smithi086 ceph-mon[38032]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 93 MiB/s wr, 14.26k op/s 2024-03-22T19:15:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:01 smithi167 ceph-mon[47728]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 93 MiB/s wr, 14.26k op/s 2024-03-22T19:15:03.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:03 smithi131 ceph-mon[78559]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 95 MiB/s wr, 14.66k op/s 2024-03-22T19:15:03.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:03 smithi086 ceph-mon[38032]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 95 MiB/s wr, 14.66k op/s 2024-03-22T19:15:03.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:03 smithi167 ceph-mon[47728]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 95 MiB/s wr, 14.66k op/s 2024-03-22T19:15:04.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:04 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:04.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:04 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:04.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:04 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:05 smithi131 ceph-mon[78559]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 100 MiB/s wr, 15.45k op/s 2024-03-22T19:15:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:05 smithi086 ceph-mon[38032]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 100 MiB/s wr, 15.45k op/s 2024-03-22T19:15:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:05 smithi167 ceph-mon[47728]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 100 MiB/s wr, 15.45k op/s 2024-03-22T19:15:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:07 smithi167 ceph-mon[47728]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 91 MiB/s wr, 14.18k op/s 2024-03-22T19:15:07.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:07 smithi131 ceph-mon[78559]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 91 MiB/s wr, 14.18k op/s 2024-03-22T19:15:07.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:07 smithi086 ceph-mon[38032]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 91 MiB/s wr, 14.18k op/s 2024-03-22T19:15:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:09 smithi131 ceph-mon[78559]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 102 MiB/s wr, 15.79k op/s 2024-03-22T19:15:09.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:09 smithi086 ceph-mon[38032]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 102 MiB/s wr, 15.79k op/s 2024-03-22T19:15:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:09 smithi167 ceph-mon[47728]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 102 MiB/s wr, 15.79k op/s 2024-03-22T19:15:11.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:11 smithi131 ceph-mon[78559]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 95 MiB/s wr, 14.34k op/s 2024-03-22T19:15:11.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:11 smithi086 ceph-mon[38032]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 95 MiB/s wr, 14.34k op/s 2024-03-22T19:15:11.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:11 smithi167 ceph-mon[47728]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 95 MiB/s wr, 14.34k op/s 2024-03-22T19:15:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:13 smithi131 ceph-mon[78559]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 97 MiB/s wr, 14.64k op/s 2024-03-22T19:15:13.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:13 smithi086 ceph-mon[38032]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 97 MiB/s wr, 14.64k op/s 2024-03-22T19:15:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:13 smithi167 ceph-mon[47728]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 97 MiB/s wr, 14.64k op/s 2024-03-22T19:15:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:14 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:14 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:14 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:14 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:14 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:14 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:14 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:14 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:14 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:15.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:15 smithi131 ceph-mon[78559]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 114 MiB/s wr, 17.27k op/s 2024-03-22T19:15:15.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:15 smithi086 ceph-mon[38032]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 114 MiB/s wr, 17.27k op/s 2024-03-22T19:15:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:15 smithi167 ceph-mon[47728]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 114 MiB/s wr, 17.27k op/s 2024-03-22T19:15:17.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:17 smithi167 ceph-mon[47728]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 96 MiB/s wr, 14.97k op/s 2024-03-22T19:15:17.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:17 smithi131 ceph-mon[78559]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 96 MiB/s wr, 14.97k op/s 2024-03-22T19:15:17.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:17 smithi086 ceph-mon[38032]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 96 MiB/s wr, 14.97k op/s 2024-03-22T19:15:19.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:19 smithi131 ceph-mon[78559]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 94 MiB/s wr, 14.95k op/s 2024-03-22T19:15:19.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:19 smithi086 ceph-mon[38032]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 94 MiB/s wr, 14.95k op/s 2024-03-22T19:15:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:19 smithi167 ceph-mon[47728]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 94 MiB/s wr, 14.95k op/s 2024-03-22T19:15:21.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:15:21.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:21 smithi131 ceph-mon[78559]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 86 MiB/s wr, 13.84k op/s 2024-03-22T19:15:21.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:21 smithi086 ceph-mon[38032]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 86 MiB/s wr, 13.84k op/s 2024-03-22T19:15:21.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:21 smithi167 ceph-mon[47728]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 86 MiB/s wr, 13.84k op/s 2024-03-22T19:15:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:23 smithi131 ceph-mon[78559]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 81 MiB/s wr, 13.09k op/s 2024-03-22T19:15:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:23 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:15:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:23 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:15:23.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:23 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:15:23.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:23 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:15:23.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:23 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:15:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:23 smithi086 ceph-mon[38032]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 81 MiB/s wr, 13.09k op/s 2024-03-22T19:15:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:23 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:15:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:23 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:15:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:23 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:15:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:23 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:15:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:23 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:15:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:23 smithi167 ceph-mon[47728]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 81 MiB/s wr, 13.09k op/s 2024-03-22T19:15:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:23 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:15:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:23 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:15:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:23 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:15:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:23 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:15:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:23 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:15:24.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:24 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:15:24.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:24 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:15:24.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:24 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:15:25.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:25 smithi131 ceph-mon[78559]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 MiB/s wr, 15.01k op/s 2024-03-22T19:15:25.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:25 smithi086 ceph-mon[38032]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 MiB/s wr, 15.01k op/s 2024-03-22T19:15:25.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:25 smithi167 ceph-mon[47728]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 MiB/s wr, 15.01k op/s 2024-03-22T19:15:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:27 smithi167 ceph-mon[47728]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 89 MiB/s wr, 13.60k op/s 2024-03-22T19:15:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 -- 172.21.15.86:0/1163620466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e81028e0 msgr2=0x7fc6e8109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 --2- 172.21.15.86:0/1163620466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e81028e0 0x7fc6e8109190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fc0 tx=0x7fc6e000b330 comp rx=0 tx=0).stop 2024-03-22T19:15:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 -- 172.21.15.86:0/1163620466 shutdown_connections 2024-03-22T19:15:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 --2- 172.21.15.86:0/1163620466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e81028e0 0x7fc6e8109190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 --2- 172.21.15.86:0/1163620466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8100d50 0x7fc6e8101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 --2- 172.21.15.86:0/1163620466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6e8101f40 0x7fc6e81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.690+0000 7fc6ef200700 1 -- 172.21.15.86:0/1163620466 >> 172.21.15.86:0/1163620466 conn(0x7fc6e80fc2c0 msgr2=0x7fc6e80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.691+0000 7fc6ef200700 1 -- 172.21.15.86:0/1163620466 shutdown_connections 2024-03-22T19:15:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.692+0000 7fc6ef200700 1 -- 172.21.15.86:0/1163620466 wait complete. 2024-03-22T19:15:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.696+0000 7fc6ef200700 1 Processor -- start 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.696+0000 7fc6ef200700 1 -- start start 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.696+0000 7fc6ef200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6e8101f40 0x7fc6e8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.696+0000 7fc6ef200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 0x7fc6e80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.696+0000 7fc6ef200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e819aa20 0x7fc6e80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.696+0000 7fc6ef200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8073e80 con 0x7fc6e81028e0 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.697+0000 7fc6ef200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8073fc0 con 0x7fc6e819aa20 2024-03-22T19:15:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.697+0000 7fc6ef200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e819ae10 con 0x7fc6e8101f40 2024-03-22T19:15:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.697+0000 7fc6e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 0x7fc6e80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.697+0000 7fc6e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 0x7fc6e80731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37094/0 (socket says 172.21.15.86:37094) 2024-03-22T19:15:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.697+0000 7fc6e7400700 1 -- 172.21.15.86:0/792549003 learned_addr learned my addr 172.21.15.86:0/792549003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.698+0000 7fc6ed200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e819aa20 0x7fc6e80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.698+0000 7fc6e7400700 1 -- 172.21.15.86:0/792549003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6e8101f40 msgr2=0x7fc6e8074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.698+0000 7fc6e7400700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6e8101f40 0x7fc6e8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.698+0000 7fc6e7400700 1 -- 172.21.15.86:0/792549003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e819aa20 msgr2=0x7fc6e80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.698+0000 7fc6e7400700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e819aa20 0x7fc6e80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.698+0000 7fc6e7400700 1 -- 172.21.15.86:0/792549003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009c40 con 0x7fc6e81028e0 2024-03-22T19:15:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.699+0000 7fc6e7400700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 0x7fc6e80731a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800d980 tx=0x7fc6d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.700+0000 7fc6e4c00700 1 -- 172.21.15.86:0/792549003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8009930 con 0x7fc6e81028e0 2024-03-22T19:15:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.700+0000 7fc6e4c00700 1 -- 172.21.15.86:0/792549003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6d8010460 con 0x7fc6e81028e0 2024-03-22T19:15:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.700+0000 7fc6e4c00700 1 -- 172.21.15.86:0/792549003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800f620 con 0x7fc6e81028e0 2024-03-22T19:15:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.700+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e819b090 con 0x7fc6e81028e0 2024-03-22T19:15:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.700+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e819b550 con 0x7fc6e81028e0 2024-03-22T19:15:27.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.712+0000 7fc6e4c00700 1 -- 172.21.15.86:0/792549003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fc6d80105d0 con 0x7fc6e81028e0 2024-03-22T19:15:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.713+0000 7fc6e4c00700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc6cc0a0b10 0x7fc6cc0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.714+0000 7fc702c31b80 1 Processor -- start 2024-03-22T19:15:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.714+0000 7fc6e4c00700 1 -- 172.21.15.86:0/792549003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc6d80cbe40 con 0x7fc6e81028e0 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.715+0000 7fc702c31b80 1 -- start start 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.715+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b1e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.715+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b1ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b1f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc702c31b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641a76990 con 0x558641b1dc20 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc702c31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641a74350 con 0x558641b1f420 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc702c31b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641a6fc40 con 0x558641b1e580 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc6cc0a0b10 0x7fc6cc0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc6e7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b1ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc6e7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b1ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:34964/0 (socket says 172.21.15.86:34964) 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc6e7400700 1 -- 172.21.15.86:0/1156550893 learned_addr learned my addr 172.21.15.86:0/1156550893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc6e7400700 1 -- 172.21.15.86:0/1156550893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 msgr2=0x558641b1f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.716+0000 7fc6ed200700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b1f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b1e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b1f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7400700 1 -- 172.21.15.86:0/1156550893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 msgr2=0x558641b1e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b1e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7400700 1 -- 172.21.15.86:0/1156550893 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586418f13b0 con 0x558641b1e580 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6ed200700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b1f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.717+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b1ee40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80a1410 tx=0x7fc6d8099100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2757a1993a0b12 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.718+0000 7fc6dd600700 1 -- 172.21.15.86:0/1156550893 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d809d2f0 con 0x558641b1e580 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.718+0000 7fc6dd600700 1 -- 172.21.15.86:0/1156550893 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6d80a1940 con 0x558641b1e580 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.718+0000 7fc6dd600700 1 -- 172.21.15.86:0/1156550893 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8098c10 con 0x558641b1e580 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.719+0000 7fc702c31b80 1 -- 172.21.15.86:0/1156550893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 msgr2=0x558641b1ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.719+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b1ee40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80a1410 tx=0x7fc6d8099100 comp rx=0 tx=0).stop 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.719+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc6cc0a0b10 0x7fc6cc0a2fd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8100890 tx=0x7fc6d40099d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 -- 172.21.15.86:0/1156550893 shutdown_connections 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b1f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1156550893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b1ee40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 -- 172.21.15.86:0/1156550893 >> 172.21.15.86:0/1156550893 conn(0x558641a93220 msgr2=0x558641b18bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:27.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:27 smithi131 ceph-mon[78559]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 89 MiB/s wr, 13.60k op/s 2024-03-22T19:15:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 -- 172.21.15.86:0/1156550893 shutdown_connections 2024-03-22T19:15:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.720+0000 7fc702c31b80 1 -- 172.21.15.86:0/1156550893 wait complete. 2024-03-22T19:15:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.722+0000 7fc702c31b80 1 -- start start 2024-03-22T19:15:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.724+0000 7fc702c31b80 1 Processor -- start 2024-03-22T19:15:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.724+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.726+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.726+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.726+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc702c31b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641a0bd80 con 0x558641b1dc20 2024-03-22T19:15:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b279b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37112/0 (socket says 172.21.15.86:37112) 2024-03-22T19:15:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc6e7e00700 1 -- 172.21.15.86:0/1302634521 learned_addr learned my addr 172.21.15.86:0/1302634521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641a0ba20 con 0x558641b1f420 2024-03-22T19:15:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586418f36a0 con 0x558641b1e580 2024-03-22T19:15:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc702c31b80 10 client.0 _ll_register_callbacks cb 0x5586418773a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.727+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6ed200700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6e7e00700 1 -- 172.21.15.86:0/1302634521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 msgr2=0x558641b27ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b27ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6e7e00700 1 -- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 msgr2=0x558641b2ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6e7e00700 1 -- 172.21.15.86:0/1302634521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e819b550 con 0x558641b1dc20 2024-03-22T19:15:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.728+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b27ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:15:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.730+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b279b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4010430 tx=0x7fc6d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c111b5f9df409d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.730+0000 7fc6d2000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:15:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.730+0000 7fc6d2000700 1 -- 172.21.15.86:0/1302634521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d401f5f0 con 0x558641b1dc20 2024-03-22T19:15:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.731+0000 7fc6d2000700 1 -- 172.21.15.86:0/1302634521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc6d401a040 con 0x558641b1dc20 2024-03-22T19:15:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.732+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558641b10ad0 con 0x558641b1dc20 2024-03-22T19:15:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.732+0000 7fc6d2000700 1 -- 172.21.15.86:0/1302634521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6d401f750 con 0x558641b1dc20 2024-03-22T19:15:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.732+0000 7fc702c31b80 10 client.16017 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.732+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55864189db60 con 0x558641b1dc20 2024-03-22T19:15:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc6d2000700 1 -- 172.21.15.86:0/1302634521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558641b10ad0 con 0x558641b1dc20 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc6d2000700 1 -- 172.21.15.86:0/1302634521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc6d4028ac0 con 0x558641b1dc20 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc702c31b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc702c31b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc702c31b80 20 client.16017 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b33790 0x558641b35bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.734+0000 7fc702c31b80 4 client.16017 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.735+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558641a68700 con 0x558641b33790 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.735+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b33790 0x558641b35bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.735+0000 7fc6e7400700 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b33790 0x558641b35bb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d595327395f51b75 server_cookie=56a318510097d9e1 in_seq=0 out_seq=0 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.736+0000 7fc6d2000700 10 client.16017 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.736+0000 7fc6d2000700 1 -- 172.21.15.86:0/1302634521 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558641a68700 con 0x558641b33790 2024-03-22T19:15:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.736+0000 7fc6d2000700 10 client.16017 handle_command_reply: tid=0 2024-03-22T19:15:27.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.737+0000 7fc702c31b80 1 client.16017 shutdown 2024-03-22T19:15:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 msgr2=0x558641b279b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b279b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4010430 tx=0x7fc6d4004750 comp rx=0 tx=0).stop 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 shutdown_connections 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b33790 0x558641b35bb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1dc20 0x558641b279b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 --2- 172.21.15.86:0/1302634521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1e580 0x558641b27ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc6d2000700 0 client.16017 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.741+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 >> 172.21.15.86:0/1302634521 conn(0x558641a93220 msgr2=0x558641b181f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.742+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 shutdown_connections 2024-03-22T19:15:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.742+0000 7fc702c31b80 1 -- 172.21.15.86:0/1302634521 wait complete. 2024-03-22T19:15:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.742+0000 7fc702c31b80 20 client.16017 trim_cache size 0 max 16384 2024-03-22T19:15:27.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:27 smithi086 ceph-mon[38032]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 89 MiB/s wr, 13.60k op/s 2024-03-22T19:15:27.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.789+0000 7fc702c31b80 1 Processor -- start 2024-03-22T19:15:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.789+0000 7fc702c31b80 1 -- start start 2024-03-22T19:15:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.789+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1dc20 0x558641b1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.789+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.789+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1f420 0x558641b181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1dc20 0x558641b1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc702c31b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641b27520 con 0x558641b1e580 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc702c31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641b26990 con 0x558641b1dc20 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc702c31b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641b12f80 con 0x558641b1f420 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1dc20 0x558641b1f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:42082/0 (socket says 172.21.15.86:42082) 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc6e7e00700 1 -- 172.21.15.86:0/2500854595 learned_addr learned my addr 172.21.15.86:0/2500854595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc6ed200700 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1f420 0x558641b181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.790+0000 7fc6e7400700 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.791+0000 7fc6ed200700 1 -- 172.21.15.86:0/2500854595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1dc20 msgr2=0x558641b1f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.791+0000 7fc6ed200700 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1dc20 0x558641b1f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.791+0000 7fc6ed200700 1 -- 172.21.15.86:0/2500854595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 msgr2=0x558641b2eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.791+0000 7fc6ed200700 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.791+0000 7fc6ed200700 1 -- 172.21.15.86:0/2500854595 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558641a35ea0 con 0x558641b1f420 2024-03-22T19:15:27.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.792+0000 7fc6ed200700 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1f420 0x558641b181f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x558641b1ec00 tx=0x7fc6e00095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ff0465eef9def0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:27.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.792+0000 7fc6de000700 1 -- 172.21.15.86:0/2500854595 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000e040 con 0x558641b1f420 2024-03-22T19:15:27.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.793+0000 7fc6de000700 1 -- 172.21.15.86:0/2500854595 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e00076e0 con 0x558641b1f420 2024-03-22T19:15:27.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.793+0000 7fc6de000700 1 -- 172.21.15.86:0/2500854595 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00052a0 con 0x558641b1f420 2024-03-22T19:15:27.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 -- 172.21.15.86:0/2500854595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1f420 msgr2=0x558641b181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1f420 0x558641b181f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x558641b1ec00 tx=0x7fc6e00095e0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 -- 172.21.15.86:0/2500854595 shutdown_connections 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1f420 0x558641b181f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1dc20 0x558641b1f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 --2- 172.21.15.86:0/2500854595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 -- 172.21.15.86:0/2500854595 >> 172.21.15.86:0/2500854595 conn(0x558641a93220 msgr2=0x558641b18b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 -- 172.21.15.86:0/2500854595 shutdown_connections 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.794+0000 7fc702c31b80 1 -- 172.21.15.86:0/2500854595 wait complete. 2024-03-22T19:15:27.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.795+0000 7fc702c31b80 1 -- start start 2024-03-22T19:15:27.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.796+0000 7fc702c31b80 1 Processor -- start 2024-03-22T19:15:27.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 0x558641b2e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641b2bd60 con 0x558641b1e580 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641b2bed0 con 0x558641b1f420 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641b2c090 con 0x558641b1dc20 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc702c31b80 10 client.0 _ll_register_callbacks cb 0x7fc6d400ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:27.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 0x558641b2e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 0x558641b2e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:34980/0 (socket says 172.21.15.86:34980) 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.797+0000 7fc6e7e00700 1 -- 172.21.15.86:0/3664134939 learned_addr learned my addr 172.21.15.86:0/3664134939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6ed200700 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6e7e00700 1 -- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 msgr2=0x558641b2b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6e7e00700 1 -- 172.21.15.86:0/3664134939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 msgr2=0x558641b2eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6e7e00700 1 -- 172.21.15.86:0/3664134939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e8109ae0 con 0x558641b1dc20 2024-03-22T19:15:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6e7e00700 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 0x558641b2e980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4010fd0 tx=0x7fc6d4027f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cafd07f7283b131a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.798+0000 7fc6d2000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:15:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.799+0000 7fc6d2000700 1 -- 172.21.15.86:0/3664134939 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4028850 con 0x558641b1dc20 2024-03-22T19:15:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.799+0000 7fc6d2000700 1 -- 172.21.15.86:0/3664134939 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fc6d40289b0 con 0x558641b1dc20 2024-03-22T19:15:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.799+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6d80a1940 con 0x558641b1dc20 2024-03-22T19:15:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.799+0000 7fc6d2000700 1 -- 172.21.15.86:0/3664134939 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6d4028b70 con 0x558641b1dc20 2024-03-22T19:15:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc702c31b80 10 client.26122 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5586418f5cd0 con 0x558641b1dc20 2024-03-22T19:15:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc6d2000700 1 -- 172.21.15.86:0/3664134939 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6d80a1940 con 0x558641b1dc20 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc6d2000700 1 -- 172.21.15.86:0/3664134939 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc6d403a030 con 0x558641b1dc20 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc702c31b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc702c31b80 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.801+0000 7fc702c31b80 20 client.26122 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.802+0000 7fc702c31b80 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b86900 0x558641b88d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.802+0000 7fc702c31b80 4 client.26122 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:15:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.802+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586418becf0 con 0x558641b86900 2024-03-22T19:15:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.802+0000 7fc6e7400700 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b86900 0x558641b88d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.803+0000 7fc6d2000700 10 client.26122 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.803+0000 7fc6e7400700 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b86900 0x558641b88d20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a973b8f1332c7db server_cookie=496306c1e3168bf4 in_seq=0 out_seq=0 2024-03-22T19:15:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.803+0000 7fc6d2000700 1 -- 172.21.15.86:0/3664134939 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5586418becf0 con 0x558641b86900 2024-03-22T19:15:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.803+0000 7fc6d2000700 10 client.26122 handle_command_reply: tid=0 2024-03-22T19:15:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.803+0000 7fc702c31b80 1 client.26122 shutdown 2024-03-22T19:15:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 msgr2=0x558641b2e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 0x558641b2e980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4010fd0 tx=0x7fc6d4027f20 comp rx=0 tx=0).stop 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 shutdown_connections 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558641b86900 0x558641b88d20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558641b1f420 0x558641b2b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558641b1dc20 0x558641b2e980 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 --2- 172.21.15.86:0/3664134939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558641b1e580 0x558641b2eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 >> 172.21.15.86:0/3664134939 conn(0x558641a93220 msgr2=0x558641b222a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.804+0000 7fc6d2000700 0 client.26122 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.805+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 shutdown_connections 2024-03-22T19:15:27.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.805+0000 7fc702c31b80 1 -- 172.21.15.86:0/3664134939 wait complete. 2024-03-22T19:15:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.810+0000 7fc702c31b80 20 client.26122 trim_cache size 0 max 16384 2024-03-22T19:15:27.812 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:15:27.812 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:15:27.812 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:15:27.812 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:15:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.811+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc6cc0a0b10 msgr2=0x7fc6cc0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.811+0000 7fc6ef200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc6cc0a0b10 0x7fc6cc0a2fd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8100890 tx=0x7fc6d40099d0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.811+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 msgr2=0x7fc6e80731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.811+0000 7fc6ef200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 0x7fc6e80731a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800d980 tx=0x7fc6d800dc90 comp rx=0 tx=0).stop 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 shutdown_connections 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fc6cc0a0b10 0x7fc6cc0a2fd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e819aa20 0x7fc6e80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6e8101f40 0x7fc6e8074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 --2- 172.21.15.86:0/792549003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e81028e0 0x7fc6e80731a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 >> 172.21.15.86:0/792549003 conn(0x7fc6e80fc2c0 msgr2=0x7fc6e8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 shutdown_connections 2024-03-22T19:15:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:27.813+0000 7fc6ef200700 1 -- 172.21.15.86:0/792549003 wait complete. 2024-03-22T19:15:27.849 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:15:27.850 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:15:28.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 -- 172.21.15.86:0/833604683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c0ff4a0 msgr2=0x7f8a8c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:28.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 --2- 172.21.15.86:0/833604683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c0ff4a0 0x7f8a8c1018d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a800098f0 tx=0x7f8a80009c00 comp rx=0 tx=0).stop 2024-03-22T19:15:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 -- 172.21.15.86:0/833604683 shutdown_connections 2024-03-22T19:15:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 --2- 172.21.15.86:0/833604683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c1047e0 0x7f8a8c104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 --2- 172.21.15.86:0/833604683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c0ff4a0 0x7f8a8c1018d0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a800098f0 tx=0x7f8a80009c00 comp rx=0 tx=0).stop 2024-03-22T19:15:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 --2- 172.21.15.86:0/833604683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c101e10 0x7f8a8c1042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.237+0000 7f8a93a00700 1 -- 172.21.15.86:0/833604683 >> 172.21.15.86:0/833604683 conn(0x7f8a8c0fb0a0 msgr2=0x7f8a8c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.238+0000 7f8a93a00700 1 -- 172.21.15.86:0/833604683 shutdown_connections 2024-03-22T19:15:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.238+0000 7f8a93a00700 1 -- 172.21.15.86:0/833604683 wait complete. 2024-03-22T19:15:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.239+0000 7f8a93a00700 1 Processor -- start 2024-03-22T19:15:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.240+0000 7f8a93a00700 1 -- start start 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.240+0000 7f8a93a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c101e10 0x7f8a8c10aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a93a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c1047e0 0x7f8a8c10b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a93a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c1ab2f0 0x7f8a8c1ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a93a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c1adc20 con 0x7f8a8c101e10 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a8be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c1047e0 0x7f8a8c10b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a93a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c1add60 con 0x7f8a8c1047e0 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a93a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c1adea0 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a91000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c101e10 0x7f8a8c10aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a8be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c1047e0 0x7f8a8c10b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:42114/0 (socket says 172.21.15.86:42114) 2024-03-22T19:15:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a91a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c1ab2f0 0x7f8a8c1ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.241+0000 7f8a8be00700 1 -- 172.21.15.86:0/555413040 learned_addr learned my addr 172.21.15.86:0/555413040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.242+0000 7f8a91a00700 1 -- 172.21.15.86:0/555413040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c1047e0 msgr2=0x7f8a8c10b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.242+0000 7f8a91a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c1047e0 0x7f8a8c10b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.242+0000 7f8a91a00700 1 -- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c101e10 msgr2=0x7f8a8c10aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.243+0000 7f8a91a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c101e10 0x7f8a8c10aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.243+0000 7f8a91a00700 1 -- 172.21.15.86:0/555413040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009520 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.243+0000 7f8a91000700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c101e10 0x7f8a8c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.243+0000 7f8a91a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c1ab2f0 0x7f8a8c1ad6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007f10 tx=0x7f8a8400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:28.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.244+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84004db0 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.244+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8400f040 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.244+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84005990 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.244+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c1ae180 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.244+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c1ae690 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.246+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c0fcc20 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.250+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f8a8400db20 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.252+0000 7f8a89600700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8a740a0c60 0x7f8a740a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.253+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f8a840cc220 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.254+0000 7f8a91000700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8a740a0c60 0x7f8a740a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.254+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a84095c60 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.257+0000 7f8a91000700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8a740a0c60 0x7f8a740a3120 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009810 tx=0x7f8a8000fcf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:28.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.772+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a8c0679e0 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.779 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:15:28.779 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:15:13.382361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:15:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.774+0000 7f8a89600700 1 -- 172.21.15.86:0/555413040 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10934 (secure 0 0 0) 0x7f8a84099280 con 0x7f8a8c1ab2f0 2024-03-22T19:15:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8a740a0c60 msgr2=0x7f8a740a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8a740a0c60 0x7f8a740a3120 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009810 tx=0x7f8a8000fcf0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c1ab2f0 msgr2=0x7f8a8c1ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c1ab2f0 0x7f8a8c1ad6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007f10 tx=0x7f8a8400d3b0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 shutdown_connections 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f8a740a0c60 0x7f8a740a3120 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a8c1ab2f0 0x7f8a8c1ad6e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8c101e10 0x7f8a8c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 --2- 172.21.15.86:0/555413040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8c1047e0 0x7f8a8c10b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 >> 172.21.15.86:0/555413040 conn(0x7f8a8c0fb0a0 msgr2=0x7f8a8c1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.780+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 shutdown_connections 2024-03-22T19:15:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:28.781+0000 7f8a93a00700 1 -- 172.21.15.86:0/555413040 wait complete. 2024-03-22T19:15:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-03-22T19:15:28.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 -- 172.21.15.86:0/2607490360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c075770 msgr2=0x7fab1c075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 --2- 172.21.15.86:0/2607490360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c075770 0x7fab1c075b90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009990 tx=0x7fab0c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 -- 172.21.15.86:0/2607490360 shutdown_connections 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 --2- 172.21.15.86:0/2607490360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 0x7fab1c077bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 --2- 172.21.15.86:0/2607490360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c075770 0x7fab1c075b90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 --2- 172.21.15.86:0/2607490360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c0769b0 0x7fab1c076e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.689+0000 7fab21000700 1 -- 172.21.15.86:0/2607490360 >> 172.21.15.86:0/2607490360 conn(0x7fab1c0fe4c0 msgr2=0x7fab1c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.692+0000 7fab21000700 1 -- 172.21.15.86:0/2607490360 shutdown_connections 2024-03-22T19:15:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.692+0000 7fab21000700 1 -- 172.21.15.86:0/2607490360 wait complete. 2024-03-22T19:15:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.694+0000 7fab21000700 1 Processor -- start 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.694+0000 7fab21000700 1 -- start start 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab21000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c075770 0x7fab1c06f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab21000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c0769b0 0x7fab1c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab21000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 0x7fab1c1ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab21000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c072cd0 con 0x7fab1c0769b0 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab21000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c072e40 con 0x7fab1c075770 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab19e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c075770 0x7fab1c06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab21000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1ad9f0 con 0x7fab1c077370 2024-03-22T19:15:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab19e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c075770 0x7fab1c06f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58136/0 (socket says 172.21.15.86:58136) 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab19e00700 1 -- 172.21.15.86:0/690259698 learned_addr learned my addr 172.21.15.86:0/690259698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.696+0000 7fab1a800700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 0x7fab1c1ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab19400700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c0769b0 0x7fab1c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab1a800700 1 -- 172.21.15.86:0/690259698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c075770 msgr2=0x7fab1c06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab1a800700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c075770 0x7fab1c06f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab1a800700 1 -- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c0769b0 msgr2=0x7fab1c06fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab1a800700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c0769b0 0x7fab1c06fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab1a800700 1 -- 172.21.15.86:0/690259698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0095f0 con 0x7fab1c077370 2024-03-22T19:15:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.697+0000 7fab1a800700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 0x7fab1c1ad4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fab1400efc0 tx=0x7fab1400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.698+0000 7fab12a00700 1 -- 172.21.15.86:0/690259698 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14018070 con 0x7fab1c077370 2024-03-22T19:15:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.698+0000 7fab12a00700 1 -- 172.21.15.86:0/690259698 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1400f040 con 0x7fab1c077370 2024-03-22T19:15:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.698+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c1adc40 con 0x7fab1c077370 2024-03-22T19:15:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.698+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1ae150 con 0x7fab1c077370 2024-03-22T19:15:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.699+0000 7fab12a00700 1 -- 172.21.15.86:0/690259698 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400cce0 con 0x7fab1c077370 2024-03-22T19:15:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.705+0000 7fab12a00700 1 -- 172.21.15.86:0/690259698 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fab14004000 con 0x7fab1c077370 2024-03-22T19:15:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.707+0000 7fab34b6fb80 1 Processor -- start 2024-03-22T19:15:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.708+0000 7fab12a00700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0b90 0x7fab000a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.708+0000 7fab12a00700 1 -- 172.21.15.86:0/690259698 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fab140cba50 con 0x7fab1c077370 2024-03-22T19:15:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.708+0000 7fab19400700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0b90 0x7fab000a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 -- start start 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601172040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x560601172e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x5606011738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606010ca990 con 0x560601171c20 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606010c8350 con 0x560601173420 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.709+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606010c3c40 con 0x560601172580 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.710+0000 7fab19400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x560601172e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.710+0000 7fab19400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x560601172e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54620/0 (socket says 172.21.15.86:54620) 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.710+0000 7fab19400700 1 -- 172.21.15.86:0/2858089480 learned_addr learned my addr 172.21.15.86:0/2858089480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.711+0000 7fab19e00700 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601172040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.711+0000 7fab1a800700 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x5606011738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.711+0000 7fab19400700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0b90 0x7fab000a3050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.712+0000 7fab19400700 1 -- 172.21.15.86:0/2858089480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 msgr2=0x5606011738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.712+0000 7fab19400700 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x5606011738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.712+0000 7fab19400700 1 -- 172.21.15.86:0/2858089480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 msgr2=0x560601172040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.712+0000 7fab19400700 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601172040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.712+0000 7fab19400700 1 -- 172.21.15.86:0/2858089480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560600f453b0 con 0x560601172580 2024-03-22T19:15:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.713+0000 7fab19400700 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x560601172e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fab0800ead0 tx=0x7fab08019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d53cea4fcf861658 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.714+0000 7fab07400700 1 -- 172.21.15.86:0/2858089480 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800eea0 con 0x560601172580 2024-03-22T19:15:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.714+0000 7fab07400700 1 -- 172.21.15.86:0/2858089480 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0801e410 con 0x560601172580 2024-03-22T19:15:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.714+0000 7fab07400700 1 -- 172.21.15.86:0/2858089480 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08016660 con 0x560601172580 2024-03-22T19:15:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.715+0000 7fab34b6fb80 1 -- 172.21.15.86:0/2858089480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 msgr2=0x560601172e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.715+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x560601172e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fab0800ead0 tx=0x7fab08019000 comp rx=0 tx=0).stop 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 -- 172.21.15.86:0/2858089480 shutdown_connections 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x5606011738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601172040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/2858089480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x560601172e40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 -- 172.21.15.86:0/2858089480 >> 172.21.15.86:0/2858089480 conn(0x5606010e7220 msgr2=0x56060116cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 -- 172.21.15.86:0/2858089480 shutdown_connections 2024-03-22T19:15:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.716+0000 7fab34b6fb80 1 -- 172.21.15.86:0/2858089480 wait complete. 2024-03-22T19:15:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.717+0000 7fab34b6fb80 1 -- start start 2024-03-22T19:15:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.718+0000 7fab34b6fb80 1 Processor -- start 2024-03-22T19:15:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.719+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x56060117b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.719+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.719+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601182e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.720+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060105fd80 con 0x560601171c20 2024-03-22T19:15:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.720+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060105fa20 con 0x560601173420 2024-03-22T19:15:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:29 smithi131 ceph-mon[78559]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 87 MiB/s wr, 12.97k op/s 2024-03-22T19:15:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:29 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/555413040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.720+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560600f476a0 con 0x560601172580 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab19400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab19400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54626/0 (socket says 172.21.15.86:54626) 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab19400700 1 -- 172.21.15.86:0/1855432409 learned_addr learned my addr 172.21.15.86:0/1855432409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab1a800700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601182e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab34b6fb80 10 client.0 _ll_register_callbacks cb 0x560600ecb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab19e00700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x56060117b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab1a800700 1 -- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 msgr2=0x56060117be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab1a800700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.721+0000 7fab1a800700 1 -- 172.21.15.86:0/1855432409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 msgr2=0x56060117b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.722+0000 7fab1a800700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x56060117b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.722+0000 7fab1a800700 1 -- 172.21.15.86:0/1855432409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0800e7b0 con 0x560601173420 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.722+0000 7fab1a800700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601182e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x560601172c00 tx=0x7fab14004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8985a370a34628a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.722+0000 7faafbe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:15:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.725+0000 7faafbe00700 1 -- 172.21.15.86:0/1855432409 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140adea0 con 0x560601173420 2024-03-22T19:15:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.726+0000 7faafbe00700 1 -- 172.21.15.86:0/1855432409 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fab1409de90 con 0x560601173420 2024-03-22T19:15:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.726+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560601164ad0 con 0x560601173420 2024-03-22T19:15:29.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.727+0000 7faafbe00700 1 -- 172.21.15.86:0/1855432409 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab140a2450 con 0x560601173420 2024-03-22T19:15:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.727+0000 7fab34b6fb80 10 client.26135 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.727+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560600ef1b60 con 0x560601173420 2024-03-22T19:15:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7faafbe00700 1 -- 172.21.15.86:0/1855432409 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560601164ad0 con 0x560601173420 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7faafbe00700 1 -- 172.21.15.86:0/1855432409 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab14099550 con 0x560601173420 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7fab34b6fb80 10 client.26135 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7fab34b6fb80 10 client.26135 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7fab34b6fb80 20 client.26135 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560601187630 0x560601189a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7fab34b6fb80 4 client.26135 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.728+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606010bc700 con 0x560601187630 2024-03-22T19:15:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.729+0000 7fab19e00700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560601187630 0x560601189a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.729+0000 7faafbe00700 10 client.26135 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:15:29.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.729+0000 7fab19e00700 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560601187630 0x560601189a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5722db3406405902 server_cookie=5b7d0299ea78f2f5 in_seq=0 out_seq=0 2024-03-22T19:15:29.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.730+0000 7faafbe00700 1 -- 172.21.15.86:0/1855432409 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5606010bc700 con 0x560601187630 2024-03-22T19:15:29.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.730+0000 7faafbe00700 10 client.26135 handle_command_reply: tid=0 2024-03-22T19:15:29.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.730+0000 7fab34b6fb80 1 client.26135 shutdown 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.733+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 msgr2=0x560601182e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.733+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601182e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x560601172c00 tx=0x7fab14004db0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 shutdown_connections 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601182e40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x56060117b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560601187630 0x560601189a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1855432409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7faafbe00700 0 client.26135 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:15:29.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 >> 172.21.15.86:0/1855432409 conn(0x5606010e7220 msgr2=0x56060116c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:29.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 shutdown_connections 2024-03-22T19:15:29.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.734+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1855432409 wait complete. 2024-03-22T19:15:29.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.735+0000 7fab34b6fb80 20 client.26135 trim_cache size 0 max 16384 2024-03-22T19:15:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:29 smithi086 ceph-mon[38032]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 87 MiB/s wr, 12.97k op/s 2024-03-22T19:15:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:29 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/555413040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.761+0000 7fab34b6fb80 1 Processor -- start 2024-03-22T19:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.761+0000 7fab34b6fb80 1 -- start start 2024-03-22T19:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601180d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601175520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560601167210 con 0x560601171c20 2024-03-22T19:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560601172960 con 0x560601173420 2024-03-22T19:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560601183fc0 con 0x560601172580 2024-03-22T19:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab19400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab19400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54634/0 (socket says 172.21.15.86:54634) 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab19400700 1 -- 172.21.15.86:0/1460116520 learned_addr learned my addr 172.21.15.86:0/1460116520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.762+0000 7fab19e00700 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601180d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab1a800700 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601175520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab19400700 1 -- 172.21.15.86:0/1460116520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 msgr2=0x560601175520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab19400700 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601175520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab19400700 1 -- 172.21.15.86:0/1460116520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 msgr2=0x560601180d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab19400700 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601180d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab19400700 1 -- 172.21.15.86:0/1460116520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560601089ea0 con 0x560601172580 2024-03-22T19:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.763+0000 7fab19400700 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117add0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab08009260 tx=0x7fab0800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7378fa65491c1088 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.764+0000 7fab07e00700 1 -- 172.21.15.86:0/1460116520 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800eea0 con 0x560601172580 2024-03-22T19:15:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.764+0000 7fab07e00700 1 -- 172.21.15.86:0/1460116520 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0801ea40 con 0x560601172580 2024-03-22T19:15:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.764+0000 7fab07e00700 1 -- 172.21.15.86:0/1460116520 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800eea0 con 0x560601172580 2024-03-22T19:15:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1460116520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 msgr2=0x56060117add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117add0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab08009260 tx=0x7fab0800e4b0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1460116520 shutdown_connections 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601173420 0x560601175520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601171c20 0x560601180d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/1460116520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601172580 0x56060117add0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.765+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1460116520 >> 172.21.15.86:0/1460116520 conn(0x5606010e7220 msgr2=0x560601172c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.766+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1460116520 shutdown_connections 2024-03-22T19:15:29.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.766+0000 7fab34b6fb80 1 -- 172.21.15.86:0/1460116520 wait complete. 2024-03-22T19:15:29.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.766+0000 7fab34b6fb80 1 -- start start 2024-03-22T19:15:29.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.767+0000 7fab34b6fb80 1 Processor -- start 2024-03-22T19:15:29.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.768+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601171c20 0x560601180960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.768+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 0x56060117ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.768+0000 7fab34b6fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601173420 0x56060117f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.768+0000 7fab19400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 0x56060117ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060117f9f0 con 0x560601172580 2024-03-22T19:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060117fb60 con 0x560601171c20 2024-03-22T19:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab34b6fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060117fd20 con 0x560601173420 2024-03-22T19:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab19e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601171c20 0x560601180960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab19400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 0x56060117ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51254/0 (socket says 172.21.15.86:51254) 2024-03-22T19:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab19400700 1 -- 172.21.15.86:0/531831680 learned_addr learned my addr 172.21.15.86:0/531831680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab1a800700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601173420 0x56060117f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab34b6fb80 10 client.0 _ll_register_callbacks cb 0x7fab14099c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab1a800700 1 -- 172.21.15.86:0/531831680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601171c20 msgr2=0x560601180960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab1a800700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601171c20 0x560601180960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab1a800700 1 -- 172.21.15.86:0/531831680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 msgr2=0x56060117ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab1a800700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 0x56060117ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab1a800700 1 -- 172.21.15.86:0/531831680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560600f453b0 con 0x560601173420 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.769+0000 7fab19400700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 0x56060117ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.770+0000 7faafbe00700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.770+0000 7fab1a800700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601173420 0x56060117f4b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fab14096ad0 tx=0x7fab140092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=206d13b2b09cada8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.770+0000 7faafbe00700 1 -- 172.21.15.86:0/531831680 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140adea0 con 0x560601173420 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.770+0000 7faafbe00700 1 -- 172.21.15.86:0/531831680 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fab1409de90 con 0x560601173420 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.770+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab0801e410 con 0x560601173420 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7faafbe00700 1 -- 172.21.15.86:0/531831680 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab140adea0 con 0x560601173420 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 10 client.26146 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560600f49cd0 con 0x560601173420 2024-03-22T19:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7faafbe00700 1 -- 172.21.15.86:0/531831680 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab14080080 con 0x560601173420 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7faafbe00700 1 -- 172.21.15.86:0/531831680 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab140b2b80 con 0x560601173420 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 10 client.26146 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 10 client.26146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 20 client.26146 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5606011da350 0x5606011dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 4 client.26146 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.771+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560600f12cf0 con 0x5606011da350 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.772+0000 7fab19e00700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5606011da350 0x5606011dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.773+0000 7faafbe00700 10 client.26146 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:15:29.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.774+0000 7fab19e00700 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5606011da350 0x5606011dc770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7d140be16a0aa40 server_cookie=ed5d106f3c5352b2 in_seq=0 out_seq=0 2024-03-22T19:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.774+0000 7faafbe00700 1 -- 172.21.15.86:0/531831680 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560600f12cf0 con 0x5606011da350 2024-03-22T19:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.774+0000 7faafbe00700 10 client.26146 handle_command_reply: tid=0 2024-03-22T19:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.774+0000 7fab34b6fb80 1 client.26146 shutdown 2024-03-22T19:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.774+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601173420 msgr2=0x56060117f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.774+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601173420 0x56060117f4b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fab14096ad0 tx=0x7fab140092e0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 shutdown_connections 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5606011da350 0x5606011dc770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560601173420 0x56060117f4b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560601171c20 0x560601180960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 --2- 172.21.15.86:0/531831680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560601172580 0x56060117ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 >> 172.21.15.86:0/531831680 conn(0x5606010e7220 msgr2=0x560601176090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7faafbe00700 0 client.26146 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:15:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 shutdown_connections 2024-03-22T19:15:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.776+0000 7fab34b6fb80 1 -- 172.21.15.86:0/531831680 wait complete. 2024-03-22T19:15:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.777+0000 7fab34b6fb80 20 client.26146 trim_cache size 0 max 16384 2024-03-22T19:15:29.779 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.779+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0b90 msgr2=0x7fab000a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.779+0000 7fab21000700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0b90 0x7fab000a3050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08009e30 comp rx=0 tx=0).stop 2024-03-22T19:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 msgr2=0x7fab1c1ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 0x7fab1c1ad4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fab1400efc0 tx=0x7fab1400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 shutdown_connections 2024-03-22T19:15:29.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0b90 0x7fab000a3050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c077370 0x7fab1c1ad4b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c075770 0x7fab1c06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.780+0000 7fab21000700 1 --2- 172.21.15.86:0/690259698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c0769b0 0x7fab1c06fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:29.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.781+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 >> 172.21.15.86:0/690259698 conn(0x7fab1c0fe4c0 msgr2=0x7fab1c0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.782+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 shutdown_connections 2024-03-22T19:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:29.782+0000 7fab21000700 1 -- 172.21.15.86:0/690259698 wait complete. 2024-03-22T19:15:29.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:15:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:29 smithi167 ceph-mon[47728]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 87 MiB/s wr, 12.97k op/s 2024-03-22T19:15:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:29 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/555413040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:15:30.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.215+0000 7f04cea00700 1 -- 172.21.15.86:0/2851053036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 msgr2=0x7f04c8108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.215+0000 7f04cea00700 1 --2- 172.21.15.86:0/2851053036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c8108cd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098f0 tx=0x7f04b8009c00 comp rx=0 tx=0).stop 2024-03-22T19:15:30.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.216+0000 7f04cea00700 1 -- 172.21.15.86:0/2851053036 shutdown_connections 2024-03-22T19:15:30.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.216+0000 7f04cea00700 1 --2- 172.21.15.86:0/2851053036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c8108cd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.216+0000 7f04cea00700 1 --2- 172.21.15.86:0/2851053036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101680 0x7f04c8103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.216+0000 7f04cea00700 1 --2- 172.21.15.86:0/2851053036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8103fb0 0x7f04c81063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.216+0000 7f04cea00700 1 -- 172.21.15.86:0/2851053036 >> 172.21.15.86:0/2851053036 conn(0x7f04c80fafb0 msgr2=0x7f04c80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.216+0000 7f04cea00700 1 -- 172.21.15.86:0/2851053036 shutdown_connections 2024-03-22T19:15:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.217+0000 7f04cea00700 1 -- 172.21.15.86:0/2851053036 wait complete. 2024-03-22T19:15:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.223+0000 7f04cea00700 1 Processor -- start 2024-03-22T19:15:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.225+0000 7f04cea00700 1 -- start start 2024-03-22T19:15:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.226+0000 7f04cea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8101680 0x7f04c8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.226+0000 7f04cea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8103fb0 0x7f04c81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.227+0000 7f04cea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c81a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04cea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81a07a0 con 0x7f04c8103fb0 2024-03-22T19:15:30.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04cea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81a0910 con 0x7f04c8101680 2024-03-22T19:15:30.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04cea00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81a0ad0 con 0x7f04c81068e0 2024-03-22T19:15:30.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04cd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8101680 0x7f04c8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ccc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8103fb0 0x7f04c81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04cd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8101680 0x7f04c8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58190/0 (socket says 172.21.15.86:58190) 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04cd600700 1 -- 172.21.15.86:0/608668073 learned_addr learned my addr 172.21.15.86:0/608668073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ce000700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c81a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ccc00700 1 -- 172.21.15.86:0/608668073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 msgr2=0x7f04c81a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ccc00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c81a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ccc00700 1 -- 172.21.15.86:0/608668073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8101680 msgr2=0x7f04c8198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ccc00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8101680 0x7f04c8198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.228+0000 7f04ccc00700 1 -- 172.21.15.86:0/608668073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b8009520 con 0x7f04c8103fb0 2024-03-22T19:15:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.229+0000 7f04ce000700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:15:30.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.232+0000 7f04ccc00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8103fb0 0x7f04c81991f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009eb0 tx=0x7f04b4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:30.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.234+0000 7f04c6000700 1 -- 172.21.15.86:0/608668073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4004220 con 0x7f04c8103fb0 2024-03-22T19:15:30.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.234+0000 7f04c6000700 1 -- 172.21.15.86:0/608668073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b4004cf0 con 0x7f04c8103fb0 2024-03-22T19:15:30.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.235+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c8109760 con 0x7f04c8103fb0 2024-03-22T19:15:30.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.235+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c8109c20 con 0x7f04c8103fb0 2024-03-22T19:15:30.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.236+0000 7f04c6000700 1 -- 172.21.15.86:0/608668073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4004220 con 0x7f04c8103fb0 2024-03-22T19:15:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.248+0000 7f04c6000700 1 -- 172.21.15.86:0/608668073 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f04b4005400 con 0x7f04c8103fb0 2024-03-22T19:15:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.249+0000 7f04e3b8bb80 1 Processor -- start 2024-03-22T19:15:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.250+0000 7f04c6000700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f04ac0a0af0 0x7f04ac0a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.250+0000 7f04c6000700 1 -- 172.21.15.86:0/608668073 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f04b40d4310 con 0x7f04c8103fb0 2024-03-22T19:15:30.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.251+0000 7f04e3b8bb80 1 -- start start 2024-03-22T19:15:30.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.252+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f420fde0 0x5564f4210220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.253+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f42114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.253+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4218270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.253+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f4168990 con 0x5564f4211020 2024-03-22T19:15:30.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.253+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f4166350 con 0x5564f420fde0 2024-03-22T19:15:30.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.253+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f4161c40 con 0x5564f42119e0 2024-03-22T19:15:30.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.254+0000 7f04cd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f420fde0 0x5564f4210220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.254+0000 7f04ce000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4218270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.254+0000 7f04ce000700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f04ac0a0af0 0x7f04ac0a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.254+0000 7f04ce000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4218270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54666/0 (socket says 172.21.15.86:54666) 2024-03-22T19:15:30.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.254+0000 7f04ce000700 1 -- 172.21.15.86:0/122818499 learned_addr learned my addr 172.21.15.86:0/122818499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:30.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.254+0000 7f04ccc00700 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f42114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.255+0000 7f04ce000700 1 -- 172.21.15.86:0/122818499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f420fde0 msgr2=0x5564f4210220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.255+0000 7f04ce000700 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f420fde0 0x5564f4210220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.255+0000 7f04ce000700 1 -- 172.21.15.86:0/122818499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 msgr2=0x5564f42114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.255+0000 7f04ce000700 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f42114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.256+0000 7f04ce000700 1 -- 172.21.15.86:0/122818499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f3fe33b0 con 0x5564f42119e0 2024-03-22T19:15:30.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.256+0000 7f04ce000700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f04ac0a0af0 0x7f04ac0a2fb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f04b8011040 tx=0x7f04b800bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:30.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.257+0000 7f04ce000700 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4218270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098f0 tx=0x7f04b8027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45de685481aeb431 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:30.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.257+0000 7f04be600700 1 -- 172.21.15.86:0/122818499 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b801c070 con 0x5564f42119e0 2024-03-22T19:15:30.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.258+0000 7f04be600700 1 -- 172.21.15.86:0/122818499 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b8005040 con 0x5564f42119e0 2024-03-22T19:15:30.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.258+0000 7f04be600700 1 -- 172.21.15.86:0/122818499 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b801c070 con 0x5564f42119e0 2024-03-22T19:15:30.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.259+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/122818499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 msgr2=0x5564f4218270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.259+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4218270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098f0 tx=0x7f04b8027000 comp rx=0 tx=0).stop 2024-03-22T19:15:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.260+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/122818499 shutdown_connections 2024-03-22T19:15:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.260+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4218270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.260+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f420fde0 0x5564f4210220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.260+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/122818499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f42114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.260+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/122818499 >> 172.21.15.86:0/122818499 conn(0x5564f4185220 msgr2=0x5564f420ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:30.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.261+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/122818499 shutdown_connections 2024-03-22T19:15:30.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.261+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/122818499 wait complete. 2024-03-22T19:15:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.261+0000 7f04e3b8bb80 1 -- start start 2024-03-22T19:15:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.262+0000 7f04e3b8bb80 1 Processor -- start 2024-03-22T19:15:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.263+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f420fde0 0x5564f4219ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.263+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f4211020 0x5564f421a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.263+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4221000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.264+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f40fdd80 con 0x5564f420fde0 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04ce000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4221000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f40fda20 con 0x5564f4211020 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f3fe56a0 con 0x5564f42119e0 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04ce000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4221000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54668/0 (socket says 172.21.15.86:54668) 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04ce000700 1 -- 172.21.15.86:0/3667810740 learned_addr learned my addr 172.21.15.86:0/3667810740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04e3b8bb80 10 client.0 _ll_register_callbacks cb 0x5564f3f693a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04cd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f420fde0 0x5564f4219ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.265+0000 7f04ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f4211020 0x5564f421a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.266+0000 7f04cd600700 1 -- 172.21.15.86:0/3667810740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 msgr2=0x5564f4221000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.266+0000 7f04cd600700 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4221000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.266+0000 7f04cd600700 1 -- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f4211020 msgr2=0x5564f421a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.266+0000 7f04cd600700 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f4211020 0x5564f421a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.266+0000 7f04cd600700 1 -- 172.21.15.86:0/3667810740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c80ff630 con 0x5564f420fde0 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.269+0000 7f04cd600700 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f420fde0 0x5564f4219ad0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f04c819a130 tx=0x7f04c0010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b15426ef529e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.269+0000 7f04b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:15:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.270+0000 7f04b2a00700 1 -- 172.21.15.86:0/3667810740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c000ea60 con 0x5564f420fde0 2024-03-22T19:15:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.270+0000 7f04b2a00700 1 -- 172.21.15.86:0/3667810740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f04c000ebc0 con 0x5564f420fde0 2024-03-22T19:15:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.270+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564f4202c30 con 0x5564f420fde0 2024-03-22T19:15:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.271+0000 7f04b2a00700 1 -- 172.21.15.86:0/3667810740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c0007e50 con 0x5564f420fde0 2024-03-22T19:15:30.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.272+0000 7f04e3b8bb80 10 client.16053 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:30.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.272+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5564f3f8fb60 con 0x5564f420fde0 2024-03-22T19:15:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.275+0000 7f04b2a00700 1 -- 172.21.15.86:0/3667810740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564f4202c30 con 0x5564f420fde0 2024-03-22T19:15:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.275+0000 7f04b2a00700 1 -- 172.21.15.86:0/3667810740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f04c0016410 con 0x5564f420fde0 2024-03-22T19:15:30.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.277+0000 7f04e3b8bb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:30.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.277+0000 7f04e3b8bb80 10 client.16053 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:15:30.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.277+0000 7f04e3b8bb80 20 client.16053 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:30.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.277+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f42258a0 0x5564f4227cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.277+0000 7f04e3b8bb80 4 client.16053 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:30.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.277+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564f415a700 con 0x5564f42258a0 2024-03-22T19:15:30.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.280+0000 7f04ccc00700 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f42258a0 0x5564f4227cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.286+0000 7f04ccc00700 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f42258a0 0x5564f4227cc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=784ec203140d6184 server_cookie=12753223ce893efe in_seq=0 out_seq=0 2024-03-22T19:15:30.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.286+0000 7f04b2a00700 10 client.16053 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:15:30.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.287+0000 7f04b2a00700 1 -- 172.21.15.86:0/3667810740 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5564f415a700 con 0x5564f42258a0 2024-03-22T19:15:30.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.287+0000 7f04b2a00700 10 client.16053 handle_command_reply: tid=0 2024-03-22T19:15:30.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 client.16053 shutdown 2024-03-22T19:15:30.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f420fde0 msgr2=0x5564f4219ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f420fde0 0x5564f4219ad0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f04c819a130 tx=0x7f04c0010de0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 shutdown_connections 2024-03-22T19:15:30.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f42258a0 0x5564f4227cc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42119e0 0x5564f4221000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f420fde0 0x5564f4219ad0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3667810740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f4211020 0x5564f421a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.292+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 >> 172.21.15.86:0/3667810740 conn(0x5564f4185220 msgr2=0x5564f420a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:30.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.293+0000 7f04b2a00700 0 client.16053 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:15:30.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.293+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 shutdown_connections 2024-03-22T19:15:30.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.293+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3667810740 wait complete. 2024-03-22T19:15:30.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.299+0000 7f04e3b8bb80 20 client.16053 trim_cache size 0 max 16384 2024-03-22T19:15:30.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.342+0000 7f04e3b8bb80 1 Processor -- start 2024-03-22T19:15:30.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.342+0000 7f04e3b8bb80 1 -- start start 2024-03-22T19:15:30.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.343+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f4227c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.343+0000 7f04cd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f4227c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.343+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.343+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f4219010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.344+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f4218e70 con 0x5564f4211020 2024-03-22T19:15:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.345+0000 7f04ce000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f4219010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.345+0000 7f04ce000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f4219010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54680/0 (socket says 172.21.15.86:54680) 2024-03-22T19:15:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.345+0000 7f04ce000700 1 -- 172.21.15.86:0/1366754601 learned_addr learned my addr 172.21.15.86:0/1366754601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.345+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f4207770 con 0x5564f42119e0 2024-03-22T19:15:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.345+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f4210e60 con 0x5564f42189c0 2024-03-22T19:15:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.346+0000 7f04cd600700 1 -- 172.21.15.86:0/1366754601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 msgr2=0x5564f4219010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.346+0000 7f04cd600700 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f4219010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.346+0000 7f04cd600700 1 -- 172.21.15.86:0/1366754601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 msgr2=0x5564f421efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.346+0000 7f04cd600700 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.346+0000 7f04cd600700 1 -- 172.21.15.86:0/1366754601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f4127ea0 con 0x5564f4211020 2024-03-22T19:15:30.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.347+0000 7f04cd600700 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f4227c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5564f420bff0 tx=0x7f04c000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3703a35cc3e4174 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:30.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.348+0000 7f04bf000700 1 -- 172.21.15.86:0/1366754601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0007c10 con 0x5564f4211020 2024-03-22T19:15:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.349+0000 7f04bf000700 1 -- 172.21.15.86:0/1366754601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c0016a90 con 0x5564f4211020 2024-03-22T19:15:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.349+0000 7f04bf000700 1 -- 172.21.15.86:0/1366754601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0015460 con 0x5564f4211020 2024-03-22T19:15:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.350+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 msgr2=0x5564f4227c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.350+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f4227c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5564f420bff0 tx=0x7f04c000e570 comp rx=0 tx=0).stop 2024-03-22T19:15:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.350+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 shutdown_connections 2024-03-22T19:15:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.351+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f4219010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.351+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f4227c90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.351+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/1366754601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.351+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 >> 172.21.15.86:0/1366754601 conn(0x5564f4185220 msgr2=0x5564f4206b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.352+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 shutdown_connections 2024-03-22T19:15:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.352+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/1366754601 wait complete. 2024-03-22T19:15:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.353+0000 7f04e3b8bb80 1 -- start start 2024-03-22T19:15:30.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.355+0000 7f04e3b8bb80 1 Processor -- start 2024-03-22T19:15:30.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.356+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f421d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.356+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.357+0000 7f04cd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f421d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.358+0000 7f04e3b8bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f421e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.358+0000 7f04e3b8bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f421ea40 con 0x5564f4211020 2024-03-22T19:15:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.358+0000 7f04cd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f421d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51316/0 (socket says 172.21.15.86:51316) 2024-03-22T19:15:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.358+0000 7f04cd600700 1 -- 172.21.15.86:0/3924053697 learned_addr learned my addr 172.21.15.86:0/3924053697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.359+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f421ebb0 con 0x5564f42119e0 2024-03-22T19:15:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.359+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f421ed20 con 0x5564f42189c0 2024-03-22T19:15:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.359+0000 7f04ccc00700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.359+0000 7f04e3b8bb80 10 client.0 _ll_register_callbacks cb 0x7f04c0010580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.359+0000 7f04ce000700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f421e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.361+0000 7f04cd600700 1 -- 172.21.15.86:0/3924053697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 msgr2=0x5564f421e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.361+0000 7f04cd600700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f421e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.361+0000 7f04cd600700 1 -- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 msgr2=0x5564f421d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.361+0000 7f04cd600700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.361+0000 7f04cd600700 1 -- 172.21.15.86:0/3924053697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564f3f8fb60 con 0x5564f4211020 2024-03-22T19:15:30.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.363+0000 7f04cd600700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f421d140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f04c0016820 tx=0x7f04c00169b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f6bf911d5bf1b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.363+0000 7f04b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:15:30.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.364+0000 7f04b2a00700 1 -- 172.21.15.86:0/3924053697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0003a00 con 0x5564f4211020 2024-03-22T19:15:30.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.366+0000 7f04b2a00700 1 -- 172.21.15.86:0/3924053697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f04c0003b60 con 0x5564f4211020 2024-03-22T19:15:30.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.366+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b8005040 con 0x5564f4211020 2024-03-22T19:15:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.368+0000 7f04b2a00700 1 -- 172.21.15.86:0/3924053697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c0003d20 con 0x5564f4211020 2024-03-22T19:15:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.368+0000 7f04e3b8bb80 10 client.16059 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.368+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5564f3fe7cd0 con 0x5564f4211020 2024-03-22T19:15:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.373+0000 7f04b2a00700 1 -- 172.21.15.86:0/3924053697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b8005040 con 0x5564f4211020 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.374+0000 7f04b2a00700 1 -- 172.21.15.86:0/3924053697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f04c003c450 con 0x5564f4211020 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.376+0000 7f04e3b8bb80 10 client.16059 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.376+0000 7f04e3b8bb80 10 client.16059 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.376+0000 7f04e3b8bb80 20 client.16059 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.376+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f4278240 0x5564f427a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.376+0000 7f04e3b8bb80 4 client.16059 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:15:30.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.376+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5564f3fb0cf0 con 0x5564f4278240 2024-03-22T19:15:30.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.386+0000 7f04ccc00700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f4278240 0x5564f427a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:30.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.397+0000 7f04ccc00700 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f4278240 0x5564f427a660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c8e63979e21fe12 server_cookie=b911e900a96ba851 in_seq=0 out_seq=0 2024-03-22T19:15:30.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.397+0000 7f04b2a00700 10 client.16059 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:15:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.399+0000 7f04b2a00700 1 -- 172.21.15.86:0/3924053697 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5564f3fb0cf0 con 0x5564f4278240 2024-03-22T19:15:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.399+0000 7f04b2a00700 10 client.16059 handle_command_reply: tid=0 2024-03-22T19:15:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.400+0000 7f04e3b8bb80 1 client.16059 shutdown 2024-03-22T19:15:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.401+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 msgr2=0x5564f421d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.401+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f421d140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f04c0016820 tx=0x7f04c00169b0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 shutdown_connections 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x5564f4278240 0x5564f427a660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04b2a00700 0 client.16059 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564f42189c0 0x5564f421e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f4211020 0x5564f421d140 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04e3b8bb80 1 --2- 172.21.15.86:0/3924053697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564f42119e0 0x5564f421d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.402+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 >> 172.21.15.86:0/3924053697 conn(0x5564f4185220 msgr2=0x5564f4218fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.403+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 shutdown_connections 2024-03-22T19:15:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.403+0000 7f04e3b8bb80 1 -- 172.21.15.86:0/3924053697 wait complete. 2024-03-22T19:15:30.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.404+0000 7f04e3b8bb80 20 client.16059 trim_cache size 0 max 16384 2024-03-22T19:15:30.407 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:15:30.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.406+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f04ac0a0af0 msgr2=0x7f04ac0a2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.406+0000 7f04cea00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f04ac0a0af0 0x7f04ac0a2fb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f04b8011040 tx=0x7f04b800bef0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.406+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8103fb0 msgr2=0x7f04c81991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:30.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.406+0000 7f04cea00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8103fb0 0x7f04c81991f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009eb0 tx=0x7f04b4009f90 comp rx=0 tx=0).stop 2024-03-22T19:15:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.407+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 shutdown_connections 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.407+0000 7f04cea00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f04ac0a0af0 0x7f04ac0a2fb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.407+0000 7f04cea00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c81068e0 0x7f04c81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.407+0000 7f04cea00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c8101680 0x7f04c8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.407+0000 7f04cea00700 1 --2- 172.21.15.86:0/608668073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8103fb0 0x7f04c81991f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.407+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 >> 172.21.15.86:0/608668073 conn(0x7f04c80fafb0 msgr2=0x7f04c8101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.408+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 shutdown_connections 2024-03-22T19:15:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:30.408+0000 7f04cea00700 1 -- 172.21.15.86:0/608668073 wait complete. 2024-03-22T19:15:30.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:15:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:31 smithi131 ceph-mon[78559]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 85 MiB/s wr, 12.44k op/s 2024-03-22T19:15:31.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:31 smithi086 ceph-mon[38032]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 85 MiB/s wr, 12.44k op/s 2024-03-22T19:15:31.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:31 smithi167 ceph-mon[47728]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 85 MiB/s wr, 12.44k op/s 2024-03-22T19:15:31.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 -- 172.21.15.86:0/2431590648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0107350 msgr2=0x7f82c0107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 --2- 172.21.15.86:0/2431590648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0107350 0x7f82c0107770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82a80099c0 tx=0x7f82a8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 -- 172.21.15.86:0/2431590648 shutdown_connections 2024-03-22T19:15:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 --2- 172.21.15.86:0/2431590648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0108ee0 0x7f82c010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 --2- 172.21.15.86:0/2431590648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0107350 0x7f82c0107770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 --2- 172.21.15.86:0/2431590648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c0108540 0x7f82c01089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.878+0000 7f82c6600700 1 -- 172.21.15.86:0/2431590648 >> 172.21.15.86:0/2431590648 conn(0x7f82c00762e0 msgr2=0x7f82c0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.879+0000 7f82c6600700 1 -- 172.21.15.86:0/2431590648 shutdown_connections 2024-03-22T19:15:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.879+0000 7f82c6600700 1 -- 172.21.15.86:0/2431590648 wait complete. 2024-03-22T19:15:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.881+0000 7f82c6600700 1 Processor -- start 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.882+0000 7f82c6600700 1 -- start start 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.882+0000 7f82c6600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0107350 0x7f82c019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.882+0000 7f82c6600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0108540 0x7f82c019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.882+0000 7f82c6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c0108ee0 0x7f82c01a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.883+0000 7f82b6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0108540 0x7f82c019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.883+0000 7f82b6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0108540 0x7f82c019d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58252/0 (socket says 172.21.15.86:58252) 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.883+0000 7f82b6a00700 1 -- 172.21.15.86:0/3783122439 learned_addr learned my addr 172.21.15.86:0/3783122439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.883+0000 7f82bf400700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0107350 0x7f82c019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.884+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0079600 con 0x7f82c0108ee0 2024-03-22T19:15:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.884+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0079770 con 0x7f82c0108540 2024-03-22T19:15:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.884+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0079930 con 0x7f82c0107350 2024-03-22T19:15:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.884+0000 7f82bfe00700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c0108ee0 0x7f82c01a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.885+0000 7f82bf400700 1 -- 172.21.15.86:0/3783122439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0108540 msgr2=0x7f82c019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.885+0000 7f82bf400700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0108540 0x7f82c019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.885+0000 7f82bf400700 1 -- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c0108ee0 msgr2=0x7f82c01a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.885+0000 7f82bf400700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c0108ee0 0x7f82c01a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.885+0000 7f82bf400700 1 -- 172.21.15.86:0/3783122439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80095f0 con 0x7f82c0107350 2024-03-22T19:15:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.892+0000 7f82bf400700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0107350 0x7f82c019d240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82a80098e0 tx=0x7f82a8003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:31.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.892+0000 7f82bcc00700 1 -- 172.21.15.86:0/3783122439 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a801d070 con 0x7f82c0107350 2024-03-22T19:15:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.893+0000 7f82bcc00700 1 -- 172.21.15.86:0/3783122439 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a800f460 con 0x7f82c0107350 2024-03-22T19:15:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.893+0000 7f82bcc00700 1 -- 172.21.15.86:0/3783122439 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8017650 con 0x7f82c0107350 2024-03-22T19:15:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.893+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c0079b50 con 0x7f82c0107350 2024-03-22T19:15:31.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.893+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c0079fd0 con 0x7f82c0107350 2024-03-22T19:15:31.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.904+0000 7f82da195b80 1 Processor -- start 2024-03-22T19:15:31.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.905+0000 7f82bcc00700 1 -- 172.21.15.86:0/3783122439 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f82a800f5d0 con 0x7f82c0107350 2024-03-22T19:15:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.905+0000 7f82da195b80 1 -- start start 2024-03-22T19:15:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.909+0000 7f82bcc00700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82a40a0c90 0x7f82a40a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.910+0000 7f82bcc00700 1 -- 172.21.15.86:0/3783122439 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f82a80cccf0 con 0x7f82c0107350 2024-03-22T19:15:31.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.911+0000 7f82bfe00700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82a40a0c90 0x7f82a40a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.912+0000 7f82da195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.912+0000 7f82da195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bc580 0x560dee6bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.912+0000 7f82da195b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.912+0000 7f82da195b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee614990 con 0x560dee6bd420 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.912+0000 7f82da195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee612350 con 0x560dee6bc580 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.912+0000 7f82da195b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee60dc40 con 0x560dee6bbc20 2024-03-22T19:15:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bfe00700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82a40a0c90 0x7f82a40a3150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82b800f3c0 tx=0x7f82b8007950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:31.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6bc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54716/0 (socket says 172.21.15.86:54716) 2024-03-22T19:15:31.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 -- 172.21.15.86:0/3570581651 learned_addr learned my addr 172.21.15.86:0/3570581651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:31.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82b6a00700 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bc580 0x560dee6bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bfe00700 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 -- 172.21.15.86:0/3570581651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bc580 msgr2=0x560dee6bce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bc580 0x560dee6bce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 -- 172.21.15.86:0/3570581651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 msgr2=0x560dee6bd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6bd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.913+0000 7f82bf400700 1 -- 172.21.15.86:0/3570581651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dee48f3b0 con 0x560dee6bbc20 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.914+0000 7f82bf400700 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6bc040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82a809a390 tx=0x7f82a8009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6940da709cc3656 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.914+0000 7f82b4c00700 1 -- 172.21.15.86:0/3570581651 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a809e030 con 0x560dee6bbc20 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.914+0000 7f82b4c00700 1 -- 172.21.15.86:0/3570581651 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a809ab00 con 0x560dee6bbc20 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.914+0000 7f82b4c00700 1 -- 172.21.15.86:0/3570581651 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800f8f0 con 0x560dee6bbc20 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 -- 172.21.15.86:0/3570581651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 msgr2=0x560dee6bc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6bc040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82a809a390 tx=0x7f82a8009cd0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 -- 172.21.15.86:0/3570581651 shutdown_connections 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6bd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6bc040 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 --2- 172.21.15.86:0/3570581651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bc580 0x560dee6bce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.915+0000 7f82da195b80 1 -- 172.21.15.86:0/3570581651 >> 172.21.15.86:0/3570581651 conn(0x560dee631220 msgr2=0x560dee6b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.916+0000 7f82da195b80 1 -- 172.21.15.86:0/3570581651 shutdown_connections 2024-03-22T19:15:31.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.917+0000 7f82da195b80 1 -- 172.21.15.86:0/3570581651 wait complete. 2024-03-22T19:15:31.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.917+0000 7f82da195b80 1 -- start start 2024-03-22T19:15:31.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.918+0000 7f82da195b80 1 Processor -- start 2024-03-22T19:15:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.923+0000 7f82da195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.923+0000 7f82da195b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bc580 0x560dee6c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.923+0000 7f82da195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.924+0000 7f82bf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.924+0000 7f82bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.924+0000 7f82bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58270/0 (socket says 172.21.15.86:58270) 2024-03-22T19:15:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.924+0000 7f82bf400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54722/0 (socket says 172.21.15.86:54722) 2024-03-22T19:15:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.925+0000 7f82bfe00700 1 -- 172.21.15.86:0/3876438616 learned_addr learned my addr 172.21.15.86:0/3876438616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.925+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee5a9d80 con 0x560dee6bc580 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.925+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee5a9a20 con 0x560dee6bd420 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.925+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee4916a0 con 0x560dee6bbc20 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82da195b80 10 client.0 _ll_register_callbacks cb 0x560dee4153a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82b6a00700 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bc580 0x560dee6c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82bf400700 1 -- 172.21.15.86:0/3876438616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 msgr2=0x560dee6ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82bf400700 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82bf400700 1 -- 172.21.15.86:0/3876438616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bc580 msgr2=0x560dee6c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82bf400700 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bc580 0x560dee6c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.926+0000 7f82bf400700 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c0079fd0 con 0x560dee6bbc20 2024-03-22T19:15:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.927+0000 7f82bf400700 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6c5700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f82a809a5c0 tx=0x7f82a809e0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d582b2719920fac server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.927+0000 7f82ad600700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:15:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.928+0000 7f82ad600700 1 -- 172.21.15.86:0/3876438616 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a80a6070 con 0x560dee6bbc20 2024-03-22T19:15:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.928+0000 7f82ad600700 1 -- 172.21.15.86:0/3876438616 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f82a80987a0 con 0x560dee6bbc20 2024-03-22T19:15:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.928+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dee6aead0 con 0x560dee6bbc20 2024-03-22T19:15:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.929+0000 7f82ad600700 1 -- 172.21.15.86:0/3876438616 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82a800fb00 con 0x560dee6bbc20 2024-03-22T19:15:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.929+0000 7f82da195b80 10 client.26173 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.929+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560dee43bb60 con 0x560dee6bbc20 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82ad600700 1 -- 172.21.15.86:0/3876438616 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dee6aead0 con 0x560dee6bbc20 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82ad600700 1 -- 172.21.15.86:0/3876438616 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f82a80a9030 con 0x560dee6bbc20 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82da195b80 10 client.26173 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82da195b80 10 client.26173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82da195b80 20 client.26173 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82da195b80 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee6d0db0 0x560dee6d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82da195b80 4 client.26173 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dee606700 con 0x560dee6d0db0 2024-03-22T19:15:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.930+0000 7f82b6a00700 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee6d0db0 0x560dee6d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.932+0000 7f82b6a00700 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee6d0db0 0x560dee6d31d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fbd5ac5f231ee38 server_cookie=507850c4d43abe in_seq=0 out_seq=0 2024-03-22T19:15:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.933+0000 7f82ad600700 10 client.26173 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:15:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.933+0000 7f82ad600700 1 -- 172.21.15.86:0/3876438616 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560dee606700 con 0x560dee6d0db0 2024-03-22T19:15:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.933+0000 7f82ad600700 10 client.26173 handle_command_reply: tid=0 2024-03-22T19:15:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.933+0000 7f82da195b80 1 client.26173 shutdown 2024-03-22T19:15:31.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.936+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 msgr2=0x560dee6c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:31.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.936+0000 7f82da195b80 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6c5700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f82a809a5c0 tx=0x7f82a809e0f0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 shutdown_connections 2024-03-22T19:15:31.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee6d0db0 0x560dee6d31d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bbc20 0x560dee6c5700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 --2- 172.21.15.86:0/3876438616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bc580 0x560dee6c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82ad600700 0 client.26173 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 >> 172.21.15.86:0/3876438616 conn(0x560dee631220 msgr2=0x560dee6b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 shutdown_connections 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.937+0000 7f82da195b80 1 -- 172.21.15.86:0/3876438616 wait complete. 2024-03-22T19:15:31.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.938+0000 7f82da195b80 20 client.26173 trim_cache size 0 max 16384 2024-03-22T19:15:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.977+0000 7f82da195b80 1 Processor -- start 2024-03-22T19:15:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.977+0000 7f82da195b80 1 -- start start 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82da195b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bbc20 0x560dee6bc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82da195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82da195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82da195b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee681940 con 0x560dee6bbc20 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82da195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee6b7930 con 0x560dee6bd420 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82da195b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee6b1210 con 0x560dee6bc580 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58272/0 (socket says 172.21.15.86:58272) 2024-03-22T19:15:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.978+0000 7f82bfe00700 1 -- 172.21.15.86:0/2459153789 learned_addr learned my addr 172.21.15.86:0/2459153789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.979+0000 7f82bfe00700 1 -- 172.21.15.86:0/2459153789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 msgr2=0x560dee6ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.979+0000 7f82b6a00700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.979+0000 7f82bf400700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bbc20 0x560dee6bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.979+0000 7f82bfe00700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.979+0000 7f82bfe00700 1 -- 172.21.15.86:0/2459153789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bbc20 msgr2=0x560dee6bc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.980+0000 7f82bfe00700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bbc20 0x560dee6bc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.980+0000 7f82bfe00700 1 -- 172.21.15.86:0/2459153789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dee5d3ea0 con 0x560dee6bd420 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.980+0000 7f82b6a00700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.980+0000 7f82bf400700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bbc20 0x560dee6bc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.980+0000 7f82bfe00700 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6c4c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82b80147d0 tx=0x7f82b8012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d8a7164421db93 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.981+0000 7f82b5600700 1 -- 172.21.15.86:0/2459153789 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800fec0 con 0x560dee6bd420 2024-03-22T19:15:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.981+0000 7f82b5600700 1 -- 172.21.15.86:0/2459153789 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b8014d40 con 0x560dee6bd420 2024-03-22T19:15:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.981+0000 7f82b5600700 1 -- 172.21.15.86:0/2459153789 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800fec0 con 0x560dee6bd420 2024-03-22T19:15:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.982+0000 7f82da195b80 1 -- 172.21.15.86:0/2459153789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 msgr2=0x560dee6c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.982+0000 7f82da195b80 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6c4c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82b80147d0 tx=0x7f82b8012000 comp rx=0 tx=0).stop 2024-03-22T19:15:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.983+0000 7f82da195b80 1 -- 172.21.15.86:0/2459153789 shutdown_connections 2024-03-22T19:15:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.983+0000 7f82da195b80 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bd420 0x560dee6c4c10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.983+0000 7f82da195b80 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bbc20 0x560dee6bc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.983+0000 7f82da195b80 1 --2- 172.21.15.86:0/2459153789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.983+0000 7f82da195b80 1 -- 172.21.15.86:0/2459153789 >> 172.21.15.86:0/2459153789 conn(0x560dee631220 msgr2=0x560dee6c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.984+0000 7f82da195b80 1 -- 172.21.15.86:0/2459153789 shutdown_connections 2024-03-22T19:15:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.984+0000 7f82da195b80 1 -- 172.21.15.86:0/2459153789 wait complete. 2024-03-22T19:15:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.985+0000 7f82da195b80 1 -- start start 2024-03-22T19:15:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.985+0000 7f82da195b80 1 Processor -- start 2024-03-22T19:15:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.987+0000 7f82da195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bbc20 0x560dee6c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.987+0000 7f82da195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.987+0000 7f82bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bbc20 0x560dee6c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bbc20 0x560dee6c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58288/0 (socket says 172.21.15.86:58288) 2024-03-22T19:15:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82da195b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82b6a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82b6a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54732/0 (socket says 172.21.15.86:54732) 2024-03-22T19:15:31.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82bf400700 1 -- 172.21.15.86:0/1188140682 learned_addr learned my addr 172.21.15.86:0/1188140682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:31.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee6cc5a0 con 0x560dee6bd420 2024-03-22T19:15:31.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82bfe00700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:31.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee6cc6e0 con 0x560dee6bbc20 2024-03-22T19:15:31.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dee6cc8a0 con 0x560dee6bc580 2024-03-22T19:15:31.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.988+0000 7f82da195b80 10 client.0 _ll_register_callbacks cb 0x7f82a8099f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:31.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.989+0000 7f82bfe00700 1 -- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 msgr2=0x560dee6c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.989+0000 7f82bfe00700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.989+0000 7f82bfe00700 1 -- 172.21.15.86:0/1188140682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bbc20 msgr2=0x560dee6c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:31.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.989+0000 7f82bfe00700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bbc20 0x560dee6c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:31.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.989+0000 7f82bfe00700 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b8014450 con 0x560dee6bd420 2024-03-22T19:15:31.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.989+0000 7f82b6a00700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:31.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.991+0000 7f82bfe00700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6cc060 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82b800fb10 tx=0x7f82b8004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61300f1eafa2a640 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:31.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.991+0000 7f82ad600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:15:31.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.992+0000 7f82ad600700 1 -- 172.21.15.86:0/1188140682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8015810 con 0x560dee6bd420 2024-03-22T19:15:31.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.992+0000 7f82ad600700 1 -- 172.21.15.86:0/1188140682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f82b8015970 con 0x560dee6bd420 2024-03-22T19:15:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.993+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82a809ab00 con 0x560dee6bd420 2024-03-22T19:15:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.993+0000 7f82ad600700 1 -- 172.21.15.86:0/1188140682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82b8015810 con 0x560dee6bd420 2024-03-22T19:15:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.993+0000 7f82da195b80 10 client.16068 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:31.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.994+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560dee493cd0 con 0x560dee6bd420 2024-03-22T19:15:31.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82ad600700 1 -- 172.21.15.86:0/1188140682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b803b030 con 0x560dee6bd420 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82ad600700 1 -- 172.21.15.86:0/1188140682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f82b80052c0 con 0x560dee6bd420 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82da195b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82da195b80 10 client.16068 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82da195b80 20 client.16068 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82da195b80 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee723870 0x560dee725c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82da195b80 4 client.16068 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.997+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560dee45ccf0 con 0x560dee723870 2024-03-22T19:15:31.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.998+0000 7f82bf400700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee723870 0x560dee725c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:32.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.999+0000 7f82bf400700 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee723870 0x560dee725c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc7fb0ed74760f68 server_cookie=f73036571fc209d6 in_seq=0 out_seq=0 2024-03-22T19:15:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.999+0000 7f82ad600700 10 client.16068 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:15:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.999+0000 7f82ad600700 1 -- 172.21.15.86:0/1188140682 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560dee45ccf0 con 0x560dee723870 2024-03-22T19:15:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.999+0000 7f82ad600700 10 client.16068 handle_command_reply: tid=0 2024-03-22T19:15:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:31.999+0000 7f82da195b80 1 client.16068 shutdown 2024-03-22T19:15:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.000+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 msgr2=0x560dee6cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:32.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.000+0000 7f82da195b80 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6cc060 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82b800fb10 tx=0x7f82b8004db0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 shutdown_connections 2024-03-22T19:15:32.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82da195b80 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x560dee723870 0x560dee725c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82da195b80 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dee6bd420 0x560dee6cc060 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82da195b80 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560dee6bbc20 0x560dee6c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82ad600700 0 client.16068 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:15:32.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82da195b80 1 --2- 172.21.15.86:0/1188140682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dee6bc580 0x560dee6c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.001+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 >> 172.21.15.86:0/1188140682 conn(0x560dee631220 msgr2=0x560dee6c4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:32.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.002+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 shutdown_connections 2024-03-22T19:15:32.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.002+0000 7f82da195b80 1 -- 172.21.15.86:0/1188140682 wait complete. 2024-03-22T19:15:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.002+0000 7f82da195b80 20 client.16068 trim_cache size 0 max 16384 2024-03-22T19:15:32.004 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:15:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82a40a0c90 msgr2=0x7f82a40a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82a40a0c90 0x7f82a40a3150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82b800f3c0 tx=0x7f82b8007950 comp rx=0 tx=0).stop 2024-03-22T19:15:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0107350 msgr2=0x7f82c019d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0107350 0x7f82c019d240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82a80098e0 tx=0x7f82a8003af0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 shutdown_connections 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f82a40a0c90 0x7f82a40a3150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c0108ee0 0x7f82c01a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c0107350 0x7f82c019d240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 --2- 172.21.15.86:0/3783122439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82c0108540 0x7f82c019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.004+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 >> 172.21.15.86:0/3783122439 conn(0x7f82c00762e0 msgr2=0x7f82c010b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.005+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 shutdown_connections 2024-03-22T19:15:32.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:32.006+0000 7f82c6600700 1 -- 172.21.15.86:0/3783122439 wait complete. 2024-03-22T19:15:32.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:15:32.419 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:15:32.419 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:15:32.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:15:32.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:15:32.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:32 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:15:32.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:32 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:15:32.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:15:32.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:15:32.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:32 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.021+0000 7f9904a00700 1 -- 172.21.15.86:0/1970195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9900108a30 msgr2=0x7f990010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 --2- 172.21.15.86:0/1970195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9900108a30 0x7f990010aea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f98f800ad00 tx=0x7f98f800c690 comp rx=0 tx=0).stop 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 -- 172.21.15.86:0/1970195265 shutdown_connections 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 --2- 172.21.15.86:0/1970195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9900108a30 0x7f990010aea0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 --2- 172.21.15.86:0/1970195265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9900074d90 0x7f99000731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 --2- 172.21.15.86:0/1970195265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99000737b0 0x7f9900073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 -- 172.21.15.86:0/1970195265 >> 172.21.15.86:0/1970195265 conn(0x7f99000fc110 msgr2=0x7f99000fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 -- 172.21.15.86:0/1970195265 shutdown_connections 2024-03-22T19:15:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.022+0000 7f9904a00700 1 -- 172.21.15.86:0/1970195265 wait complete. 2024-03-22T19:15:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.024+0000 7f9904a00700 1 Processor -- start 2024-03-22T19:15:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.025+0000 7f9904a00700 1 -- start start 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.026+0000 7f9904a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99000737b0 0x7f9900194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f9904a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 0x7f9900194e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f9904a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9900108a30 0x7f990019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f9904a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990019c470 con 0x7f99000737b0 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f98fd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 0x7f9900194e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f9904a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99001ad690 con 0x7f9900074d90 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f9904a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99001ad7d0 con 0x7f9900108a30 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f98fd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 0x7f9900194e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58294/0 (socket says 172.21.15.86:58294) 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.027+0000 7f98fd400700 1 -- 172.21.15.86:0/899751326 learned_addr learned my addr 172.21.15.86:0/899751326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fde00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99000737b0 0x7f9900194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fd400700 1 -- 172.21.15.86:0/899751326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9900108a30 msgr2=0x7f990019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fd400700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9900108a30 0x7f990019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fd400700 1 -- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99000737b0 msgr2=0x7f9900194940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fd400700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99000737b0 0x7f9900194940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fd400700 1 -- 172.21.15.86:0/899751326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f800a930 con 0x7f9900074d90 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fde00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99000737b0 0x7f9900194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.028+0000 7f98fd400700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 0x7f9900194e80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b530 tx=0x7f98ec00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.029+0000 7f98f6a00700 1 -- 172.21.15.86:0/899751326 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec011830 con 0x7f9900074d90 2024-03-22T19:15:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.029+0000 7f98f6a00700 1 -- 172.21.15.86:0/899751326 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98ec011e50 con 0x7f9900074d90 2024-03-22T19:15:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.029+0000 7f98f6a00700 1 -- 172.21.15.86:0/899751326 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec010e70 con 0x7f9900074d90 2024-03-22T19:15:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.029+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99001ada00 con 0x7f9900074d90 2024-03-22T19:15:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.030+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99001adec0 con 0x7f9900074d90 2024-03-22T19:15:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.036+0000 7f98f6a00700 1 -- 172.21.15.86:0/899751326 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f98ec01a400 con 0x7f9900074d90 2024-03-22T19:15:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.043+0000 7f98f6a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98e40a0c90 0x7f98e40a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.043+0000 7f98f6a00700 1 -- 172.21.15.86:0/899751326 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f98ec0cb730 con 0x7f9900074d90 2024-03-22T19:15:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.045+0000 7f98fde00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98e40a0c90 0x7f98e40a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.046+0000 7f98fde00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98e40a0c90 0x7f98e40a3150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98f0005b40 tx=0x7f98f0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.047+0000 7f991849fb80 1 Processor -- start 2024-03-22T19:15:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.048+0000 7f991849fb80 1 -- start start 2024-03-22T19:15:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.048+0000 7f991849fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a556de0 0x55de8a557220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.048+0000 7f991849fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a5584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.048+0000 7f991849fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 0x55de8a55f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.048+0000 7f991849fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a4af990 con 0x55de8a556de0 2024-03-22T19:15:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.049+0000 7f991849fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a4ad350 con 0x55de8a558020 2024-03-22T19:15:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.049+0000 7f991849fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a4a8c40 con 0x55de8a5589e0 2024-03-22T19:15:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.051+0000 7f98ff200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 0x55de8a55f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.051+0000 7f98ff200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 0x55de8a55f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54760/0 (socket says 172.21.15.86:54760) 2024-03-22T19:15:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.051+0000 7f98ff200700 1 -- 172.21.15.86:0/3411661122 learned_addr learned my addr 172.21.15.86:0/3411661122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.051+0000 7f98fde00700 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a556de0 0x55de8a557220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.052+0000 7f98fd400700 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a5584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.052+0000 7f98ff200700 1 -- 172.21.15.86:0/3411661122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 msgr2=0x55de8a5584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.052+0000 7f98ff200700 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a5584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.052+0000 7f98ff200700 1 -- 172.21.15.86:0/3411661122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a556de0 msgr2=0x55de8a557220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.052+0000 7f98ff200700 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a556de0 0x55de8a557220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.052+0000 7f98ff200700 1 -- 172.21.15.86:0/3411661122 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de8a32a3b0 con 0x55de8a5589e0 2024-03-22T19:15:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.053+0000 7f98ff200700 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 0x55de8a55f270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98f8001530 tx=0x7f98f8001560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad52f33f0c764c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.053+0000 7f98eb000700 1 -- 172.21.15.86:0/3411661122 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f801b070 con 0x55de8a5589e0 2024-03-22T19:15:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.054+0000 7f98eb000700 1 -- 172.21.15.86:0/3411661122 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f80013a0 con 0x55de8a5589e0 2024-03-22T19:15:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.054+0000 7f98eb000700 1 -- 172.21.15.86:0/3411661122 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f801b070 con 0x55de8a5589e0 2024-03-22T19:15:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 -- 172.21.15.86:0/3411661122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 msgr2=0x55de8a55f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 0x55de8a55f270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98f8001530 tx=0x7f98f8001560 comp rx=0 tx=0).stop 2024-03-22T19:15:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 -- 172.21.15.86:0/3411661122 shutdown_connections 2024-03-22T19:15:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a5589e0 0x55de8a55f270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a556de0 0x55de8a557220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 --2- 172.21.15.86:0/3411661122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a5584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.055+0000 7f991849fb80 1 -- 172.21.15.86:0/3411661122 >> 172.21.15.86:0/3411661122 conn(0x55de8a4cc220 msgr2=0x55de8a551e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.058+0000 7f991849fb80 1 -- 172.21.15.86:0/3411661122 shutdown_connections 2024-03-22T19:15:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.058+0000 7f991849fb80 1 -- 172.21.15.86:0/3411661122 wait complete. 2024-03-22T19:15:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.058+0000 7f991849fb80 1 -- start start 2024-03-22T19:15:33.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.059+0000 7f991849fb80 1 Processor -- start 2024-03-22T19:15:33.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.061+0000 7f991849fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a560a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f991849fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a558020 0x55de8a560f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f991849fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 0x55de8a567fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f991849fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a444d80 con 0x55de8a558020 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f991849fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a444a20 con 0x55de8a5589e0 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f991849fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a32c6a0 con 0x55de8a556de0 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f991849fb80 10 client.0 _ll_register_callbacks cb 0x55de8a2b03a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f98ff200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 0x55de8a567fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f98ff200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 0x55de8a567fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58320/0 (socket says 172.21.15.86:58320) 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.064+0000 7f98ff200700 1 -- 172.21.15.86:0/519293115 learned_addr learned my addr 172.21.15.86:0/519293115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.065+0000 7f98fd400700 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a558020 0x55de8a560f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.065+0000 7f98ff200700 1 -- 172.21.15.86:0/519293115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 msgr2=0x55de8a560a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.065+0000 7f98ff200700 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a560a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.065+0000 7f98ff200700 1 -- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a558020 msgr2=0x55de8a560f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.065+0000 7f98ff200700 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a558020 0x55de8a560f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.065+0000 7f98ff200700 1 -- 172.21.15.86:0/519293115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990010bac0 con 0x55de8a5589e0 2024-03-22T19:15:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.067+0000 7f98ff200700 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 0x55de8a567fa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98f8010040 tx=0x7f98f80169f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397a87b0d0db8a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.067+0000 7f98df400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:15:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.070+0000 7f98df400700 1 -- 172.21.15.86:0/519293115 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f801b040 con 0x55de8a5589e0 2024-03-22T19:15:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.070+0000 7f98df400700 1 -- 172.21.15.86:0/519293115 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f98f801f410 con 0x55de8a5589e0 2024-03-22T19:15:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.070+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de8a549c30 con 0x55de8a5589e0 2024-03-22T19:15:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.073+0000 7f98df400700 1 -- 172.21.15.86:0/519293115 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f801f5d0 con 0x55de8a5589e0 2024-03-22T19:15:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.074+0000 7f991849fb80 10 client.26183 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.074+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55de8a2d6b60 con 0x55de8a5589e0 2024-03-22T19:15:33.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.078+0000 7f98df400700 1 -- 172.21.15.86:0/519293115 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de8a549c30 con 0x55de8a5589e0 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.078+0000 7f98df400700 1 -- 172.21.15.86:0/519293115 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f98f8026070 con 0x55de8a5589e0 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.079+0000 7f991849fb80 10 client.26183 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.079+0000 7f991849fb80 10 client.26183 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.079+0000 7f991849fb80 20 client.26183 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.079+0000 7f991849fb80 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a56c3c0 0x55de8a56e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.079+0000 7f991849fb80 4 client.26183 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.079+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de8a4a1700 con 0x55de8a56c3c0 2024-03-22T19:15:33.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.080+0000 7f98fde00700 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a56c3c0 0x55de8a56e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.083+0000 7f98fde00700 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a56c3c0 0x55de8a56e7e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef3cb563b86220cb server_cookie=6c9f8fa68ca3714b in_seq=0 out_seq=0 2024-03-22T19:15:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.083+0000 7f98df400700 10 client.26183 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:15:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.087+0000 7f98df400700 1 -- 172.21.15.86:0/519293115 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55de8a4a1700 con 0x55de8a56c3c0 2024-03-22T19:15:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.088+0000 7f98df400700 10 client.26183 handle_command_reply: tid=0 2024-03-22T19:15:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.088+0000 7f991849fb80 1 client.26183 shutdown 2024-03-22T19:15:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.088+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 msgr2=0x55de8a567fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.088+0000 7f991849fb80 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 0x55de8a567fa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98f8010040 tx=0x7f98f80169f0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.089+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 shutdown_connections 2024-03-22T19:15:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.089+0000 7f991849fb80 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a56c3c0 0x55de8a56e7e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.089+0000 7f991849fb80 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a5589e0 0x55de8a567fa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.089+0000 7f991849fb80 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a560a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.089+0000 7f991849fb80 1 --2- 172.21.15.86:0/519293115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a558020 0x55de8a560f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.089+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 >> 172.21.15.86:0/519293115 conn(0x55de8a4cc220 msgr2=0x55de8a551350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.090+0000 7f98df400700 0 client.26183 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:15:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.090+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 shutdown_connections 2024-03-22T19:15:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.090+0000 7f991849fb80 1 -- 172.21.15.86:0/519293115 wait complete. 2024-03-22T19:15:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.090+0000 7f991849fb80 20 client.26183 trim_cache size 0 max 16384 2024-03-22T19:15:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.130+0000 7f991849fb80 1 Processor -- start 2024-03-22T19:15:33.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.131+0000 7f991849fb80 1 -- start start 2024-03-22T19:15:33.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.132+0000 7f991849fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a565f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.132+0000 7f991849fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a558400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.133+0000 7f991849fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a5571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.133+0000 7f991849fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a558dc0 con 0x55de8a5589e0 2024-03-22T19:15:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.133+0000 7f991849fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a561c10 con 0x55de8a558020 2024-03-22T19:15:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.133+0000 7f991849fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a5579b0 con 0x55de8a556de0 2024-03-22T19:15:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.133+0000 7f98fde00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a565f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.134+0000 7f98ff200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a5571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.134+0000 7f98ff200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a5571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51412/0 (socket says 172.21.15.86:51412) 2024-03-22T19:15:33.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.134+0000 7f98ff200700 1 -- 172.21.15.86:0/1053995192 learned_addr learned my addr 172.21.15.86:0/1053995192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.135+0000 7f98ff200700 1 -- 172.21.15.86:0/1053995192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 msgr2=0x55de8a565f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.135+0000 7f98ff200700 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a565f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.135+0000 7f98ff200700 1 -- 172.21.15.86:0/1053995192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 msgr2=0x55de8a558400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.135+0000 7f98ff200700 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a558400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.135+0000 7f98ff200700 1 -- 172.21.15.86:0/1053995192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de8a46eea0 con 0x55de8a5589e0 2024-03-22T19:15:33.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.136+0000 7f98ff200700 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a5571c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f98f800ac20 tx=0x7f98f800d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b971179e79d2b47 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.137+0000 7f98eba00700 1 -- 172.21.15.86:0/1053995192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f802fe10 con 0x55de8a5589e0 2024-03-22T19:15:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.137+0000 7f98eba00700 1 -- 172.21.15.86:0/1053995192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f80010f0 con 0x55de8a5589e0 2024-03-22T19:15:33.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.137+0000 7f98eba00700 1 -- 172.21.15.86:0/1053995192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f802fe10 con 0x55de8a5589e0 2024-03-22T19:15:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.138+0000 7f991849fb80 1 -- 172.21.15.86:0/1053995192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 msgr2=0x55de8a5571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.138+0000 7f991849fb80 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a5571c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f98f800ac20 tx=0x7f98f800d8c0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.139+0000 7f991849fb80 1 -- 172.21.15.86:0/1053995192 shutdown_connections 2024-03-22T19:15:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.139+0000 7f991849fb80 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a5571c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.139+0000 7f991849fb80 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a556de0 0x55de8a565f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.139+0000 7f991849fb80 1 --2- 172.21.15.86:0/1053995192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a558020 0x55de8a558400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.140+0000 7f991849fb80 1 -- 172.21.15.86:0/1053995192 >> 172.21.15.86:0/1053995192 conn(0x55de8a4cc220 msgr2=0x55de8a55ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.141+0000 7f991849fb80 1 -- 172.21.15.86:0/1053995192 shutdown_connections 2024-03-22T19:15:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.141+0000 7f991849fb80 1 -- 172.21.15.86:0/1053995192 wait complete. 2024-03-22T19:15:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.142+0000 7f991849fb80 1 -- start start 2024-03-22T19:15:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.142+0000 7f991849fb80 1 Processor -- start 2024-03-22T19:15:33.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.146+0000 7f991849fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a556de0 0x55de8a563be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.146+0000 7f991849fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a558020 0x55de8a564120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.146+0000 7f991849fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a565090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.146+0000 7f991849fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a5655d0 con 0x55de8a5589e0 2024-03-22T19:15:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.146+0000 7f991849fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a565740 con 0x55de8a556de0 2024-03-22T19:15:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.146+0000 7f991849fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8a565900 con 0x55de8a558020 2024-03-22T19:15:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.147+0000 7f991849fb80 10 client.0 _ll_register_callbacks cb 0x7f98f00098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.147+0000 7f98fde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a556de0 0x55de8a563be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.148+0000 7f98fde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a556de0 0x55de8a563be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58332/0 (socket says 172.21.15.86:58332) 2024-03-22T19:15:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.148+0000 7f98fde00700 1 -- 172.21.15.86:0/808226833 learned_addr learned my addr 172.21.15.86:0/808226833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.147+0000 7f98fd400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a558020 0x55de8a564120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.148+0000 7f98ff200700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a565090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.149+0000 7f98fd400700 1 -- 172.21.15.86:0/808226833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a556de0 msgr2=0x55de8a563be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.149+0000 7f98fd400700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a556de0 0x55de8a563be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.149+0000 7f98fd400700 1 -- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 msgr2=0x55de8a565090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.149+0000 7f98fd400700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a565090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.149+0000 7f98fd400700 1 -- 172.21.15.86:0/808226833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de8a2d6b60 con 0x55de8a558020 2024-03-22T19:15:33.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.150+0000 7f98ff200700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a565090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.153+0000 7f98fd400700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a558020 0x55de8a564120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0c9220 tx=0x7f98ec098710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bd1e4adf2b1827c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.153+0000 7f98df400700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:15:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.154+0000 7f98df400700 1 -- 172.21.15.86:0/808226833 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec09cbe0 con 0x55de8a558020 2024-03-22T19:15:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.154+0000 7f98df400700 1 -- 172.21.15.86:0/808226833 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f98ec0a1230 con 0x55de8a558020 2024-03-22T19:15:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.154+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98f80013a0 con 0x55de8a558020 2024-03-22T19:15:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.154+0000 7f98df400700 1 -- 172.21.15.86:0/808226833 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98ec0b71b0 con 0x55de8a558020 2024-03-22T19:15:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.155+0000 7f991849fb80 10 client.26182 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.155+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55de8a32ecd0 con 0x55de8a558020 2024-03-22T19:15:33.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f98df400700 1 -- 172.21.15.86:0/808226833 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98ec0a5020 con 0x55de8a558020 2024-03-22T19:15:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f98df400700 1 -- 172.21.15.86:0/808226833 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f98ec0982a0 con 0x55de8a558020 2024-03-22T19:15:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f991849fb80 10 client.26182 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f991849fb80 10 client.26182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:15:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f991849fb80 20 client.26182 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f991849fb80 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a5bed90 0x55de8a5c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f991849fb80 4 client.26182 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:15:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.156+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de8a2f7cf0 con 0x55de8a5bed90 2024-03-22T19:15:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.157+0000 7f98fde00700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a5bed90 0x55de8a5c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.158+0000 7f98fde00700 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a5bed90 0x55de8a5c11b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=572c25e8ca2a905e server_cookie=aba8dcc907cd6a5f in_seq=0 out_seq=0 2024-03-22T19:15:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.159+0000 7f98df400700 10 client.26182 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:15:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.159+0000 7f98df400700 1 -- 172.21.15.86:0/808226833 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de8a2f7cf0 con 0x55de8a5bed90 2024-03-22T19:15:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.159+0000 7f98df400700 10 client.26182 handle_command_reply: tid=0 2024-03-22T19:15:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.159+0000 7f991849fb80 1 client.26182 shutdown 2024-03-22T19:15:33.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.160+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a558020 msgr2=0x55de8a564120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.160+0000 7f991849fb80 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a558020 0x55de8a564120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0c9220 tx=0x7f98ec098710 comp rx=0 tx=0).stop 2024-03-22T19:15:33.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 shutdown_connections 2024-03-22T19:15:33.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f991849fb80 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55de8a5bed90 0x55de8a5c11b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f991849fb80 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8a5589e0 0x55de8a565090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f991849fb80 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8a556de0 0x55de8a563be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f991849fb80 1 --2- 172.21.15.86:0/808226833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8a558020 0x55de8a564120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f98df400700 0 client.26182 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:15:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.162+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 >> 172.21.15.86:0/808226833 conn(0x55de8a4cc220 msgr2=0x55de8a5609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.164+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 shutdown_connections 2024-03-22T19:15:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.164+0000 7f991849fb80 1 -- 172.21.15.86:0/808226833 wait complete. 2024-03-22T19:15:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.164+0000 7f991849fb80 20 client.26182 trim_cache size 0 max 16384 2024-03-22T19:15:33.166 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.165+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98e40a0c90 msgr2=0x7f98e40a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.165+0000 7f9904a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98e40a0c90 0x7f98e40a3150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98f0005b40 tx=0x7f98f0005a90 comp rx=0 tx=0).stop 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.165+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 msgr2=0x7f9900194e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.165+0000 7f9904a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 0x7f9900194e80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b530 tx=0x7f98ec00b8f0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 shutdown_connections 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f98e40a0c90 0x7f98e40a3150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9900108a30 0x7f990019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99000737b0 0x7f9900194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 --2- 172.21.15.86:0/899751326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9900074d90 0x7f9900194e80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 >> 172.21.15.86:0/899751326 conn(0x7f99000fc110 msgr2=0x7f9900100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 shutdown_connections 2024-03-22T19:15:33.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.166+0000 7f9904a00700 1 -- 172.21.15.86:0/899751326 wait complete. 2024-03-22T19:15:33.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:15:33.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:33 smithi086 ceph-mon[38032]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 88 MiB/s wr, 12.63k op/s 2024-03-22T19:15:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 -- 172.21.15.86:0/1421337690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 msgr2=0x7fa914101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 --2- 172.21.15.86:0/1421337690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 0x7fa914101160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa904009990 tx=0x7fa904009ca0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 -- 172.21.15.86:0/1421337690 shutdown_connections 2024-03-22T19:15:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 --2- 172.21.15.86:0/1421337690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa914109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 --2- 172.21.15.86:0/1421337690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 0x7fa914101160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 --2- 172.21.15.86:0/1421337690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa914101f30 0x7fa914102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.579+0000 7fa918a00700 1 -- 172.21.15.86:0/1421337690 >> 172.21.15.86:0/1421337690 conn(0x7fa9140fc2b0 msgr2=0x7fa9140fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.584+0000 7fa918a00700 1 -- 172.21.15.86:0/1421337690 shutdown_connections 2024-03-22T19:15:33.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.584+0000 7fa918a00700 1 -- 172.21.15.86:0/1421337690 wait complete. 2024-03-22T19:15:33.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.587+0000 7fa918a00700 1 Processor -- start 2024-03-22T19:15:33.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.587+0000 7fa918a00700 1 -- start start 2024-03-22T19:15:33.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.589+0000 7fa918a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 0x7fa914194a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.589+0000 7fa918a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa914101f30 0x7fa914194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.589+0000 7fa918a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa9141ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.589+0000 7fa918a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914197fd0 con 0x7fa914101f30 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.589+0000 7fa918a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914198140 con 0x7fa9141028d0 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.589+0000 7fa918a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914198300 con 0x7fa914100d40 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa9141ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa9141ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58364/0 (socket says 172.21.15.86:58364) 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 -- 172.21.15.86:0/3531195496 learned_addr learned my addr 172.21.15.86:0/3531195496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa911e00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 0x7fa914194a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 -- 172.21.15.86:0/3531195496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 msgr2=0x7fa914194a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 0x7fa914194a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 -- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa914101f30 msgr2=0x7fa914194f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa914101f30 0x7fa914194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.590+0000 7fa912800700 1 -- 172.21.15.86:0/3531195496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040095f0 con 0x7fa9141028d0 2024-03-22T19:15:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.591+0000 7fa912800700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa9141ad480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa90c007d70 tx=0x7fa90c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.591+0000 7fa90aa00700 1 -- 172.21.15.86:0/3531195496 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00ad40 con 0x7fa9141028d0 2024-03-22T19:15:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.591+0000 7fa90aa00700 1 -- 172.21.15.86:0/3531195496 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90c0128e0 con 0x7fa9141028d0 2024-03-22T19:15:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.591+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9141adb10 con 0x7fa9141028d0 2024-03-22T19:15:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.591+0000 7fa90aa00700 1 -- 172.21.15.86:0/3531195496 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0116f0 con 0x7fa9141028d0 2024-03-22T19:15:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.592+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141ae020 con 0x7fa9141028d0 2024-03-22T19:15:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.605+0000 7fa90aa00700 1 -- 172.21.15.86:0/3531195496 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa90c012420 con 0x7fa9141028d0 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.606+0000 7fa92c43fb80 1 Processor -- start 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 -- start start 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad48101040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48101e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad481028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad48059990 con 0x55ad48101580 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad48057350 con 0x55ad48100c20 2024-03-22T19:15:33.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.607+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad48052c40 con 0x55ad48102420 2024-03-22T19:15:33.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.608+0000 7fa911400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad48101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.608+0000 7fa911400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad481028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.608+0000 7fa911400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad481028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54834/0 (socket says 172.21.15.86:54834) 2024-03-22T19:15:33.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.608+0000 7fa911400700 1 -- 172.21.15.86:0/1233389339 learned_addr learned my addr 172.21.15.86:0/1233389339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.608+0000 7fa911e00700 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.610+0000 7fa90aa00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8f80a0b40 0x7fa8f80a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.610+0000 7fa90aa00700 1 -- 172.21.15.86:0/3531195496 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa90c0cc330 con 0x7fa9141028d0 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.612+0000 7fa911400700 1 -- 172.21.15.86:0/1233389339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 msgr2=0x55ad48101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.612+0000 7fa911400700 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad48101040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.612+0000 7fa911400700 1 -- 172.21.15.86:0/1233389339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 msgr2=0x55ad48101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.612+0000 7fa911400700 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48101e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.612+0000 7fa911400700 1 -- 172.21.15.86:0/1233389339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad47ed43b0 con 0x55ad48102420 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.613+0000 7fa912800700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8f80a0b40 0x7fa8f80a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.613+0000 7fa911400700 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad481028a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9000138c0 tx=0x7fa900005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3a405123c10b16 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.614+0000 7fa8ff000700 1 -- 172.21.15.86:0/1233389339 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900004a30 con 0x55ad48102420 2024-03-22T19:15:33.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.614+0000 7fa8ff000700 1 -- 172.21.15.86:0/1233389339 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9000054e0 con 0x55ad48102420 2024-03-22T19:15:33.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.615+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1233389339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 msgr2=0x55ad481028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.615+0000 7fa8ff000700 1 -- 172.21.15.86:0/1233389339 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900004a30 con 0x55ad48102420 2024-03-22T19:15:33.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.615+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad481028a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9000138c0 tx=0x7fa900005c50 comp rx=0 tx=0).stop 2024-03-22T19:15:33.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.616+0000 7fa912800700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8f80a0b40 0x7fa8f80a3000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0a19a0 tx=0x7fa90c0b9040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.616+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1233389339 shutdown_connections 2024-03-22T19:15:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.616+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad481028a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.616+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad48101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.616+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1233389339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.616+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1233389339 >> 172.21.15.86:0/1233389339 conn(0x55ad48076220 msgr2=0x55ad480fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.617+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1233389339 shutdown_connections 2024-03-22T19:15:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.617+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1233389339 wait complete. 2024-03-22T19:15:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.617+0000 7fa92c43fb80 1 -- start start 2024-03-22T19:15:33.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.618+0000 7fa92c43fb80 1 Processor -- start 2024-03-22T19:15:33.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.618+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48100c20 0x55ad4810a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.618+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 0x55ad4810adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa911400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48100c20 0x55ad4810a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48102420 0x55ad48111e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad47feed80 con 0x55ad48102420 2024-03-22T19:15:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad47feea20 con 0x55ad48101580 2024-03-22T19:15:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad47ed66a0 con 0x55ad48100c20 2024-03-22T19:15:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa911e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 0x55ad4810adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa911e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 0x55ad4810adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58378/0 (socket says 172.21.15.86:58378) 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa911e00700 1 -- 172.21.15.86:0/282198132 learned_addr learned my addr 172.21.15.86:0/282198132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa911400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48100c20 0x55ad4810a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54848/0 (socket says 172.21.15.86:54848) 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.619+0000 7fa92c43fb80 10 client.0 _ll_register_callbacks cb 0x55ad47e5a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.620+0000 7fa911e00700 1 -- 172.21.15.86:0/282198132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48100c20 msgr2=0x55ad4810a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.620+0000 7fa911e00700 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48100c20 0x55ad4810a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.620+0000 7fa911e00700 1 -- 172.21.15.86:0/282198132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48102420 msgr2=0x55ad48111e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.620+0000 7fa911e00700 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48102420 0x55ad48111e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.620+0000 7fa911e00700 1 -- 172.21.15.86:0/282198132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa900013570 con 0x55ad48101580 2024-03-22T19:15:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.620+0000 7fa8f3400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:15:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.621+0000 7fa911e00700 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 0x55ad4810adf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa904009990 tx=0x7fa904009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dcad8cc0994c7d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.622+0000 7fa8f3400700 1 -- 172.21.15.86:0/282198132 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9040050c0 con 0x55ad48101580 2024-03-22T19:15:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.622+0000 7fa8f3400700 1 -- 172.21.15.86:0/282198132 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa904005220 con 0x55ad48101580 2024-03-22T19:15:33.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.622+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad480f3ad0 con 0x55ad48101580 2024-03-22T19:15:33.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.623+0000 7fa8f3400700 1 -- 172.21.15.86:0/282198132 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9040050c0 con 0x55ad48101580 2024-03-22T19:15:33.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.623+0000 7fa92c43fb80 10 client.26195 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:33.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.623+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ad47e80b60 con 0x55ad48101580 2024-03-22T19:15:33.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.623+0000 7fa8f3400700 1 -- 172.21.15.86:0/282198132 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad480f3ad0 con 0x55ad48101580 2024-03-22T19:15:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.624+0000 7fa8f3400700 1 -- 172.21.15.86:0/282198132 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa90400fab0 con 0x55ad48101580 2024-03-22T19:15:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.625+0000 7fa92c43fb80 10 client.26195 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.625+0000 7fa92c43fb80 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.625+0000 7fa92c43fb80 20 client.26195 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.625+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48116690 0x55ad48118ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.625+0000 7fa92c43fb80 4 client.26195 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.625+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad4804b700 con 0x55ad48116690 2024-03-22T19:15:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.626+0000 7fa911400700 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48116690 0x55ad48118ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.627+0000 7fa911400700 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48116690 0x55ad48118ab0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba628c4fce4c4b80 server_cookie=10bca83e60a0f063 in_seq=0 out_seq=0 2024-03-22T19:15:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.627+0000 7fa8f3400700 10 client.26195 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.628+0000 7fa8f3400700 1 -- 172.21.15.86:0/282198132 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ad4804b700 con 0x55ad48116690 2024-03-22T19:15:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.628+0000 7fa8f3400700 10 client.26195 handle_command_reply: tid=0 2024-03-22T19:15:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.628+0000 7fa92c43fb80 1 client.26195 shutdown 2024-03-22T19:15:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 msgr2=0x55ad4810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 0x55ad4810adf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa904009990 tx=0x7fa904009ee0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 shutdown_connections 2024-03-22T19:15:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48116690 0x55ad48118ab0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48102420 0x55ad48111e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48100c20 0x55ad4810a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/282198132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48101580 0x55ad4810adf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.629+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 >> 172.21.15.86:0/282198132 conn(0x55ad48076220 msgr2=0x55ad480fb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.630+0000 7fa8f3400700 0 client.26195 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.630+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 shutdown_connections 2024-03-22T19:15:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.630+0000 7fa92c43fb80 1 -- 172.21.15.86:0/282198132 wait complete. 2024-03-22T19:15:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.631+0000 7fa92c43fb80 20 client.26195 trim_cache size 0 max 16384 2024-03-22T19:15:33.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 Processor -- start 2024-03-22T19:15:33.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 -- start start 2024-03-22T19:15:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad4810fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48109e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad48101da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad480f5fc0 con 0x55ad48101580 2024-03-22T19:15:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad481128a0 con 0x55ad48100c20 2024-03-22T19:15:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.657+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad48105c70 con 0x55ad48102420 2024-03-22T19:15:33.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad4810fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad4810fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58380/0 (socket says 172.21.15.86:58380) 2024-03-22T19:15:33.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 -- 172.21.15.86:0/1877377476 learned_addr learned my addr 172.21.15.86:0/1877377476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad48101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 -- 172.21.15.86:0/1877377476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 msgr2=0x55ad48101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad48101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.658+0000 7fa911400700 1 -- 172.21.15.86:0/1877377476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 msgr2=0x55ad48109e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.659+0000 7fa911e00700 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48109e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.660+0000 7fa911400700 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48109e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.660+0000 7fa911400700 1 -- 172.21.15.86:0/1877377476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad48018ea0 con 0x55ad48100c20 2024-03-22T19:15:33.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.661+0000 7fa911e00700 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:15:33.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.661+0000 7fa911400700 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad4810fde0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa900013eb0 tx=0x7fa900005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb44b37d29b89c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.661+0000 7fa8ffa00700 1 -- 172.21.15.86:0/1877377476 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000b440 con 0x55ad48100c20 2024-03-22T19:15:33.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.662+0000 7fa8ffa00700 1 -- 172.21.15.86:0/1877377476 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa900024040 con 0x55ad48100c20 2024-03-22T19:15:33.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.662+0000 7fa8ffa00700 1 -- 172.21.15.86:0/1877377476 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900026de0 con 0x55ad48100c20 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1877377476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 msgr2=0x55ad4810fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad4810fde0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa900013eb0 tx=0x7fa900005d20 comp rx=0 tx=0).stop 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1877377476 shutdown_connections 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48102420 0x55ad48101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48100c20 0x55ad4810fde0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/1877377476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48101580 0x55ad48109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.663+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1877377476 >> 172.21.15.86:0/1877377476 conn(0x55ad48076220 msgr2=0x55ad48108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.664+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1877377476 shutdown_connections 2024-03-22T19:15:33.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.664+0000 7fa92c43fb80 1 -- 172.21.15.86:0/1877377476 wait complete. 2024-03-22T19:15:33.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.665+0000 7fa92c43fb80 1 -- start start 2024-03-22T19:15:33.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.665+0000 7fa92c43fb80 1 Processor -- start 2024-03-22T19:15:33.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48100c20 0x55ad4810f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 0x55ad4810fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48102420 0x55ad4810dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4810e320 con 0x55ad48100c20 2024-03-22T19:15:33.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4810e490 con 0x55ad48102420 2024-03-22T19:15:33.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4810e650 con 0x55ad48101580 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.666+0000 7fa92c43fb80 10 client.0 _ll_register_callbacks cb 0x7fa90400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 0x55ad4810fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 0x55ad4810fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54872/0 (socket says 172.21.15.86:54872) 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 -- 172.21.15.86:0/3171495051 learned_addr learned my addr 172.21.15.86:0/3171495051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 -- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48102420 msgr2=0x55ad4810dd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48102420 0x55ad4810dd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 -- 172.21.15.86:0/3171495051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48100c20 msgr2=0x55ad4810f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48100c20 0x55ad4810f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.667+0000 7fa911e00700 1 -- 172.21.15.86:0/3171495051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad47ed43b0 con 0x55ad48101580 2024-03-22T19:15:33.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.668+0000 7fa911e00700 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 0x55ad4810fbe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa904005850 tx=0x7fa904003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45fa7aea674e9404 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:33.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.668+0000 7fa8f3400700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:15:33.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.669+0000 7fa8f3400700 1 -- 172.21.15.86:0/3171495051 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9040050b0 con 0x55ad48101580 2024-03-22T19:15:33.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.669+0000 7fa8f3400700 1 -- 172.21.15.86:0/3171495051 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa904005220 con 0x55ad48101580 2024-03-22T19:15:33.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.670+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9000054e0 con 0x55ad48101580 2024-03-22T19:15:33.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.670+0000 7fa8f3400700 1 -- 172.21.15.86:0/3171495051 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa904005900 con 0x55ad48101580 2024-03-22T19:15:33.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.670+0000 7fa92c43fb80 10 client.26200 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:33.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.670+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ad47ed8cd0 con 0x55ad48101580 2024-03-22T19:15:33.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.670+0000 7fa8f3400700 1 -- 172.21.15.86:0/3171495051 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90403c070 con 0x55ad48101580 2024-03-22T19:15:33.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.670+0000 7fa8f3400700 1 -- 172.21.15.86:0/3171495051 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa90400f660 con 0x55ad48101580 2024-03-22T19:15:33.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa92c43fb80 10 client.26200 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:33.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa92c43fb80 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa92c43fb80 20 client.26200 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48169050 0x55ad4816b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa92c43fb80 4 client.26200 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:15:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad47ea1cf0 con 0x55ad48169050 2024-03-22T19:15:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.672+0000 7fa911400700 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48169050 0x55ad4816b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa911400700 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48169050 0x55ad4816b440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e678294c1e0fd8 server_cookie=62a373e5a78f9e43 in_seq=0 out_seq=0 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa8f3400700 10 client.26200 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa8f3400700 1 -- 172.21.15.86:0/3171495051 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad47ea1cf0 con 0x55ad48169050 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa8f3400700 10 client.26200 handle_command_reply: tid=0 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa92c43fb80 1 client.26200 shutdown 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 msgr2=0x55ad4810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.673+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 0x55ad4810fbe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa904005850 tx=0x7fa904003680 comp rx=0 tx=0).stop 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 shutdown_connections 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55ad48169050 0x55ad4816b440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa8f3400700 0 client.26200 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad48102420 0x55ad4810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad48100c20 0x55ad4810f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa92c43fb80 1 --2- 172.21.15.86:0/3171495051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad48101580 0x55ad4810fbe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.674+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 >> 172.21.15.86:0/3171495051 conn(0x55ad48076220 msgr2=0x55ad48108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.675+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 shutdown_connections 2024-03-22T19:15:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.675+0000 7fa92c43fb80 1 -- 172.21.15.86:0/3171495051 wait complete. 2024-03-22T19:15:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.676+0000 7fa92c43fb80 20 client.26200 trim_cache size 0 max 16384 2024-03-22T19:15:33.678 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:15:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.677+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8f80a0b40 msgr2=0x7fa8f80a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.677+0000 7fa918a00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8f80a0b40 0x7fa8f80a3000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0a19a0 tx=0x7fa90c0b9040 comp rx=0 tx=0).stop 2024-03-22T19:15:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.677+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 msgr2=0x7fa9141ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.677+0000 7fa918a00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa9141ad480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa90c007d70 tx=0x7fa90c00a340 comp rx=0 tx=0).stop 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 shutdown_connections 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8f80a0b40 0x7fa8f80a3000 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9141028d0 0x7fa9141ad480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa914100d40 0x7fa914194a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 --2- 172.21.15.86:0/3531195496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa914101f30 0x7fa914194f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 >> 172.21.15.86:0/3531195496 conn(0x7fa9140fc2b0 msgr2=0x7fa9141051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 shutdown_connections 2024-03-22T19:15:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:33.678+0000 7fa918a00700 1 -- 172.21.15.86:0/3531195496 wait complete. 2024-03-22T19:15:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:33 smithi131 ceph-mon[78559]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 88 MiB/s wr, 12.63k op/s 2024-03-22T19:15:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:33 smithi167 ceph-mon[47728]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 88 MiB/s wr, 12.63k op/s 2024-03-22T19:15:34.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:15:34.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:15:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.048+0000 7f931ee00700 1 -- 172.21.15.86:0/4212714674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 msgr2=0x7f931810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.048+0000 7f931ee00700 1 --2- 172.21.15.86:0/4212714674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f931810c820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93040099c0 tx=0x7f9304009cd0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.049+0000 7f931ee00700 1 -- 172.21.15.86:0/4212714674 shutdown_connections 2024-03-22T19:15:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.049+0000 7f931ee00700 1 --2- 172.21.15.86:0/4212714674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931810ce40 0x7f9318117cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.049+0000 7f931ee00700 1 --2- 172.21.15.86:0/4212714674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9318072b50 0x7f9318109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.049+0000 7f931ee00700 1 --2- 172.21.15.86:0/4212714674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f931810c820 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.049+0000 7f931ee00700 1 -- 172.21.15.86:0/4212714674 >> 172.21.15.86:0/4212714674 conn(0x7f931806da20 msgr2=0x7f931806fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.051+0000 7f931ee00700 1 -- 172.21.15.86:0/4212714674 shutdown_connections 2024-03-22T19:15:35.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.051+0000 7f931ee00700 1 -- 172.21.15.86:0/4212714674 wait complete. 2024-03-22T19:15:35.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.052+0000 7f931ee00700 1 Processor -- start 2024-03-22T19:15:35.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 -- start start 2024-03-22T19:15:35.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9318072b50 0x7f93181142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f93181147f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931810ce40 0x7f93181b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318117880 con 0x7f931810ce40 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93181179f0 con 0x7f931810a390 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.054+0000 7f931ee00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318117bb0 con 0x7f9318072b50 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.055+0000 7f931d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f93181147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.055+0000 7f931d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f93181147f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58412/0 (socket says 172.21.15.86:58412) 2024-03-22T19:15:35.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.055+0000 7f931d000700 1 -- 172.21.15.86:0/3213173494 learned_addr learned my addr 172.21.15.86:0/3213173494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931da00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9318072b50 0x7f93181142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931e400700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931810ce40 0x7f93181b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931d000700 1 -- 172.21.15.86:0/3213173494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9318072b50 msgr2=0x7f93181142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931d000700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9318072b50 0x7f93181142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931d000700 1 -- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931810ce40 msgr2=0x7f93181b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931d000700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931810ce40 0x7f93181b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.056+0000 7f931d000700 1 -- 172.21.15.86:0/3213173494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040095f0 con 0x7f931810a390 2024-03-22T19:15:35.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.057+0000 7f931d000700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f93181147f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9304000c00 tx=0x7f9304005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.057+0000 7f9316000700 1 -- 172.21.15.86:0/3213173494 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930401d070 con 0x7f931810a390 2024-03-22T19:15:35.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.058+0000 7f9316000700 1 -- 172.21.15.86:0/3213173494 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93040041a0 con 0x7f931810a390 2024-03-22T19:15:35.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.058+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93181b6150 con 0x7f931810a390 2024-03-22T19:15:35.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.058+0000 7f9316000700 1 -- 172.21.15.86:0/3213173494 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400f460 con 0x7f931810a390 2024-03-22T19:15:35.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.058+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93181b6570 con 0x7f931810a390 2024-03-22T19:15:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.067+0000 7f9316000700 1 -- 172.21.15.86:0/3213173494 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f930400f5c0 con 0x7f931810a390 2024-03-22T19:15:35.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.069+0000 7f9333f01b80 1 Processor -- start 2024-03-22T19:15:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 -- start start 2024-03-22T19:15:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ce60 0x55d6deb1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb1e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ea60 0x55d6deb252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6dea75a10 con 0x55d6deb1ea60 2024-03-22T19:15:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6dea733d0 con 0x55d6deb1ce60 2024-03-22T19:15:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.070+0000 7f9333f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6dea6ecc0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.071+0000 7f931da00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb1e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.071+0000 7f931e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ce60 0x55d6deb1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.071+0000 7f931e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ce60 0x55d6deb1d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58414/0 (socket says 172.21.15.86:58414) 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.071+0000 7f931e400700 1 -- 172.21.15.86:0/386308380 learned_addr learned my addr 172.21.15.86:0/386308380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.071+0000 7f9316000700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f92fc0a0be0 0x7f92fc0a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.071+0000 7f931d000700 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ea60 0x55d6deb252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931da00700 1 -- 172.21.15.86:0/386308380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ce60 msgr2=0x55d6deb1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931da00700 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ce60 0x55d6deb1d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931da00700 1 -- 172.21.15.86:0/386308380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ea60 msgr2=0x55d6deb252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f9316000700 1 -- 172.21.15.86:0/3213173494 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f93040cd4f0 con 0x7f931810a390 2024-03-22T19:15:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931da00700 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ea60 0x55d6deb252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931da00700 1 -- 172.21.15.86:0/386308380 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6de8f0430 con 0x55d6deb1e0a0 2024-03-22T19:15:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931d000700 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ea60 0x55d6deb252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931e400700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f92fc0a0be0 0x7f92fc0a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.072+0000 7f931da00700 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb1e520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9310007fb0 tx=0x7f931000e440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ca6fd2dbc0c416 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.073+0000 7f930ea00700 1 -- 172.21.15.86:0/386308380 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000ed80 con 0x55d6deb1e0a0 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.073+0000 7f930ea00700 1 -- 172.21.15.86:0/386308380 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9310004bb0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.073+0000 7f930ea00700 1 -- 172.21.15.86:0/386308380 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310004710 con 0x55d6deb1e0a0 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.074+0000 7f9333f01b80 1 -- 172.21.15.86:0/386308380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 msgr2=0x55d6deb1e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.074+0000 7f9333f01b80 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb1e520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9310007fb0 tx=0x7f931000e440 comp rx=0 tx=0).stop 2024-03-22T19:15:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.074+0000 7f931e400700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f92fc0a0be0 0x7f92fc0a30a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f930800afc0 tx=0x7f930800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.075+0000 7f9333f01b80 1 -- 172.21.15.86:0/386308380 shutdown_connections 2024-03-22T19:15:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.075+0000 7f9333f01b80 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ea60 0x55d6deb252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.075+0000 7f9333f01b80 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ce60 0x55d6deb1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.075+0000 7f9333f01b80 1 --2- 172.21.15.86:0/386308380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb1e520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.075+0000 7f9333f01b80 1 -- 172.21.15.86:0/386308380 >> 172.21.15.86:0/386308380 conn(0x55d6dea922a0 msgr2=0x55d6deb17e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.076+0000 7f9333f01b80 1 -- 172.21.15.86:0/386308380 shutdown_connections 2024-03-22T19:15:35.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.076+0000 7f9333f01b80 1 -- 172.21.15.86:0/386308380 wait complete. 2024-03-22T19:15:35.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.077+0000 7f9333f01b80 1 -- start start 2024-03-22T19:15:35.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.077+0000 7f9333f01b80 1 Processor -- start 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.078+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb26cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.079+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.079+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.079+0000 7f9333f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6dea0ae00 con 0x55d6deb1e0a0 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.079+0000 7f9333f01b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6dea0aaa0 con 0x55d6deb1ea60 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.079+0000 7f9333f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de8f2720 con 0x55d6deb1ce60 2024-03-22T19:15:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.079+0000 7f9333f01b80 10 client.0 _ll_register_callbacks cb 0x55d6de876420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.080+0000 7f931d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.080+0000 7f931e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.080+0000 7f931e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb2e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58430/0 (socket says 172.21.15.86:58430) 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.080+0000 7f931e400700 1 -- 172.21.15.86:0/1866170271 learned_addr learned my addr 172.21.15.86:0/1866170271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.080+0000 7f931da00700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb26cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931d000700 1 -- 172.21.15.86:0/1866170271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 msgr2=0x55d6deb26cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931d000700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb26cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931d000700 1 -- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 msgr2=0x55d6deb2e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931d000700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb2e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931d000700 1 -- 172.21.15.86:0/1866170271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931000f040 con 0x55d6deb1e0a0 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931da00700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb26cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.081+0000 7f931e400700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb2e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.084+0000 7f931d000700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb271f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9304098610 tx=0x7f930409c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665965ee5501f206 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.084+0000 7f9303400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:15:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.085+0000 7f9303400700 1 -- 172.21.15.86:0/1866170271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040cd2d0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.085+0000 7f9303400700 1 -- 172.21.15.86:0/1866170271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f930409bba0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.085+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6deb0fcb0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.085+0000 7f9303400700 1 -- 172.21.15.86:0/1866170271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930409f9d0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.085+0000 7f9333f01b80 10 client.16089 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.085+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d6de89cbe0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9303400700 1 -- 172.21.15.86:0/1866170271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6deb0fcb0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9303400700 1 -- 172.21.15.86:0/1866170271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9304097990 con 0x55d6deb1e0a0 2024-03-22T19:15:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9333f01b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9333f01b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9333f01b80 20 client.16089 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb325a0 0x55d6deb349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9333f01b80 4 client.16089 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.086+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6dea67780 con 0x55d6deb325a0 2024-03-22T19:15:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.087+0000 7f931da00700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb325a0 0x55d6deb349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.088+0000 7f9303400700 10 client.16089 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.088+0000 7f931da00700 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb325a0 0x55d6deb349c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e5b09cc5c83042 server_cookie=53cd417641d5ec50 in_seq=0 out_seq=0 2024-03-22T19:15:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.089+0000 7f9303400700 1 -- 172.21.15.86:0/1866170271 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d6dea67780 con 0x55d6deb325a0 2024-03-22T19:15:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.089+0000 7f9303400700 10 client.16089 handle_command_reply: tid=0 2024-03-22T19:15:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.089+0000 7f9333f01b80 1 client.16089 shutdown 2024-03-22T19:15:35.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.091+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 msgr2=0x55d6deb271f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.091+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb271f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9304098610 tx=0x7f930409c0e0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.091+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 shutdown_connections 2024-03-22T19:15:35.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.091+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb325a0 0x55d6deb349c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.092+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb2e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.092+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb26cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.092+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1866170271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb271f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.092+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 >> 172.21.15.86:0/1866170271 conn(0x55d6dea922a0 msgr2=0x55d6deb173d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.092+0000 7f9303400700 0 client.16089 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.092+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 shutdown_connections 2024-03-22T19:15:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.093+0000 7f9333f01b80 1 -- 172.21.15.86:0/1866170271 wait complete. 2024-03-22T19:15:35.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.095+0000 7f9333f01b80 20 client.16089 trim_cache size 0 max 16384 2024-03-22T19:15:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 Processor -- start 2024-03-22T19:15:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 -- start start 2024-03-22T19:15:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ce60 0x55d6deb2c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb26cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6deb1cb00 con 0x55d6deb1ce60 2024-03-22T19:15:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6deb147c0 con 0x55d6deb1ea60 2024-03-22T19:15:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.133+0000 7f9333f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6deb26820 con 0x55d6deb1e0a0 2024-03-22T19:15:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.134+0000 7f931e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.134+0000 7f931e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb271f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58444/0 (socket says 172.21.15.86:58444) 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.134+0000 7f931e400700 1 -- 172.21.15.86:0/2864086599 learned_addr learned my addr 172.21.15.86:0/2864086599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.134+0000 7f931da00700 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ce60 0x55d6deb2c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.134+0000 7f931d000700 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb26cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.135+0000 7f931d000700 1 -- 172.21.15.86:0/2864086599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 msgr2=0x55d6deb271f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.135+0000 7f931d000700 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb271f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.135+0000 7f931d000700 1 -- 172.21.15.86:0/2864086599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ce60 msgr2=0x55d6deb2c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.135+0000 7f931d000700 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ce60 0x55d6deb2c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.135+0000 7f931d000700 1 -- 172.21.15.86:0/2864086599 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6dea34f20 con 0x55d6deb1e0a0 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.135+0000 7f931da00700 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ce60 0x55d6deb2c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.136+0000 7f931d000700 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb26cb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9304099ef0 tx=0x7f9304097db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e64d8746407cbf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.136+0000 7f930f400700 1 -- 172.21.15.86:0/2864086599 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930409bba0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.136+0000 7f930f400700 1 -- 172.21.15.86:0/2864086599 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f930409f9d0 con 0x55d6deb1e0a0 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.136+0000 7f930f400700 1 -- 172.21.15.86:0/2864086599 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040a3d60 con 0x55d6deb1e0a0 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 -- 172.21.15.86:0/2864086599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 msgr2=0x55d6deb26cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb26cb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9304099ef0 tx=0x7f9304097db0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 -- 172.21.15.86:0/2864086599 shutdown_connections 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb271f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1ce60 0x55d6deb2c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 --2- 172.21.15.86:0/2864086599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1e0a0 0x55d6deb26cb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 -- 172.21.15.86:0/2864086599 >> 172.21.15.86:0/2864086599 conn(0x55d6dea922a0 msgr2=0x55d6deb21910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 -- 172.21.15.86:0/2864086599 shutdown_connections 2024-03-22T19:15:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.137+0000 7f9333f01b80 1 -- 172.21.15.86:0/2864086599 wait complete. 2024-03-22T19:15:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.138+0000 7f9333f01b80 1 -- start start 2024-03-22T19:15:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.138+0000 7f9333f01b80 1 Processor -- start 2024-03-22T19:15:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.139+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb7c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.140+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb7cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.140+0000 7f9333f01b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb83dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.140+0000 7f931da00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb7c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f931da00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb7c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54922/0 (socket says 172.21.15.86:54922) 2024-03-22T19:15:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f931da00700 1 -- 172.21.15.86:0/1928955720 learned_addr learned my addr 172.21.15.86:0/1928955720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f931e400700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb83dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6deb84310 con 0x55d6deb1e0a0 2024-03-22T19:15:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6deb84450 con 0x55d6deb1ea60 2024-03-22T19:15:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6deb845e0 con 0x55d6deb1ce60 2024-03-22T19:15:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.141+0000 7f9333f01b80 10 client.0 _ll_register_callbacks cb 0x55d6de876420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.142+0000 7f931e400700 1 -- 172.21.15.86:0/1928955720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 msgr2=0x55d6deb7c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.142+0000 7f931e400700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb7c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.142+0000 7f931e400700 1 -- 172.21.15.86:0/1928955720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 msgr2=0x55d6deb7cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.142+0000 7f931e400700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb7cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.142+0000 7f931e400700 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6de8f0430 con 0x55d6deb1ea60 2024-03-22T19:15:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.142+0000 7f931da00700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb7c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.143+0000 7f9303400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:15:35.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.143+0000 7f931e400700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb83dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f930800a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b2e784be336516 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.144+0000 7f9303400700 1 -- 172.21.15.86:0/1928955720 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308015b50 con 0x55d6deb1ea60 2024-03-22T19:15:35.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.144+0000 7f9303400700 1 -- 172.21.15.86:0/1928955720 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f930801b040 con 0x55d6deb1ea60 2024-03-22T19:15:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.145+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9310004bb0 con 0x55d6deb1ea60 2024-03-22T19:15:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.145+0000 7f9303400700 1 -- 172.21.15.86:0/1928955720 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9308016e60 con 0x55d6deb1ea60 2024-03-22T19:15:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.145+0000 7f9333f01b80 10 client.26216 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.145+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d6de89cbe0 con 0x55d6deb1ea60 2024-03-22T19:15:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9303400700 1 -- 172.21.15.86:0/1928955720 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9310004bb0 con 0x55d6deb1ea60 2024-03-22T19:15:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9303400700 1 -- 172.21.15.86:0/1928955720 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9308028020 con 0x55d6deb1ea60 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9333f01b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9333f01b80 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9333f01b80 20 client.26216 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb85fb0 0x55d6deb883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9333f01b80 4 client.26216 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.146+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d6de8bdd70 con 0x55d6deb85fb0 2024-03-22T19:15:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.147+0000 7f931da00700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb85fb0 0x55d6deb883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.148+0000 7f931da00700 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb85fb0 0x55d6deb883a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa54a7e9ae79c5c server_cookie=ed6f26d583043fe6 in_seq=0 out_seq=0 2024-03-22T19:15:35.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.148+0000 7f9303400700 10 client.26216 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.149+0000 7f9303400700 1 -- 172.21.15.86:0/1928955720 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d6de8bdd70 con 0x55d6deb85fb0 2024-03-22T19:15:35.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.149+0000 7f9303400700 10 client.26216 handle_command_reply: tid=0 2024-03-22T19:15:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.149+0000 7f9333f01b80 1 client.26216 shutdown 2024-03-22T19:15:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 msgr2=0x55d6deb83dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb83dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f930800a160 comp rx=0 tx=0).stop 2024-03-22T19:15:35.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 shutdown_connections 2024-03-22T19:15:35.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55d6deb85fb0 0x55d6deb883a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6deb1ea60 0x55d6deb83dd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6deb1ce60 0x55d6deb7c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 --2- 172.21.15.86:0/1928955720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6deb1e0a0 0x55d6deb7cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 >> 172.21.15.86:0/1928955720 conn(0x55d6dea922a0 msgr2=0x55d6deb26c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9303400700 0 client.26216 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.150+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 shutdown_connections 2024-03-22T19:15:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.151+0000 7f9333f01b80 1 -- 172.21.15.86:0/1928955720 wait complete. 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.151+0000 7f9333f01b80 20 client.26216 trim_cache size 0 max 16384 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.152+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f92fc0a0be0 msgr2=0x7f92fc0a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.152+0000 7f931ee00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f92fc0a0be0 0x7f92fc0a30a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f930800afc0 tx=0x7f930800a4c0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.152+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 msgr2=0x7f93181147f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.152+0000 7f931ee00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f93181147f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9304000c00 tx=0x7f9304005810 comp rx=0 tx=0).stop 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:15:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:15:35.156 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:15:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 shutdown_connections 2024-03-22T19:15:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f92fc0a0be0 0x7f92fc0a30a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931810ce40 0x7f93181b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9318072b50 0x7f93181142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 --2- 172.21.15.86:0/3213173494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f931810a390 0x7f93181147f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 >> 172.21.15.86:0/3213173494 conn(0x7f931806da20 msgr2=0x7f9318109b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 shutdown_connections 2024-03-22T19:15:35.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.154+0000 7f931ee00700 1 -- 172.21.15.86:0/3213173494 wait complete. 2024-03-22T19:15:35.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:15:35.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:35 smithi131 ceph-mon[78559]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 87 MiB/s wr, 12.30k op/s 2024-03-22T19:15:35.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:35 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:15:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:35 smithi086 ceph-mon[38032]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 87 MiB/s wr, 12.30k op/s 2024-03-22T19:15:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:35 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:15:35.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:35 smithi167 ceph-mon[47728]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 87 MiB/s wr, 12.30k op/s 2024-03-22T19:15:35.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:35 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:15:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.885+0000 7f8549c00700 1 -- 172.21.15.86:0/2939217944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544100d40 msgr2=0x7f8544101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.885+0000 7f8549c00700 1 --2- 172.21.15.86:0/2939217944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544100d40 0x7f8544101160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f853400b0d0 tx=0x7f853400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 -- 172.21.15.86:0/2939217944 shutdown_connections 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 --2- 172.21.15.86:0/2939217944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 0x7f8544109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 --2- 172.21.15.86:0/2939217944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544100d40 0x7f8544101160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 --2- 172.21.15.86:0/2939217944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8544101f30 0x7f8544102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 -- 172.21.15.86:0/2939217944 >> 172.21.15.86:0/2939217944 conn(0x7f85440fc2f0 msgr2=0x7f85440fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 -- 172.21.15.86:0/2939217944 shutdown_connections 2024-03-22T19:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.886+0000 7f8549c00700 1 -- 172.21.15.86:0/2939217944 wait complete. 2024-03-22T19:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.888+0000 7f8549c00700 1 Processor -- start 2024-03-22T19:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 -- start start 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8544100d40 0x7f854410ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544101f30 0x7f8544110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 0x7f85441ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544110c10 con 0x7f8544100d40 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85441ad980 con 0x7f8544101f30 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.889+0000 7f8549c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85441adaf0 con 0x7f85441028d0 2024-03-22T19:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.890+0000 7f8543400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 0x7f85441ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.890+0000 7f8543400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 0x7f85441ad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54936/0 (socket says 172.21.15.86:54936) 2024-03-22T19:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.890+0000 7f8543400700 1 -- 172.21.15.86:0/667281506 learned_addr learned my addr 172.21.15.86:0/667281506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.890+0000 7f853a000700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544101f30 0x7f8544110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.891+0000 7f853a000700 1 -- 172.21.15.86:0/667281506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 msgr2=0x7f85441ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.891+0000 7f853a000700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 0x7f85441ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.891+0000 7f853a000700 1 -- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8544100d40 msgr2=0x7f854410ffc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:15:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.891+0000 7f8542a00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8544100d40 0x7f854410ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.891+0000 7f853a000700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8544100d40 0x7f854410ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.891+0000 7f853a000700 1 -- 172.21.15.86:0/667281506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534009d00 con 0x7f8544101f30 2024-03-22T19:15:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.892+0000 7f853a000700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544101f30 0x7f8544110500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f852c00d750 tx=0x7f852c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.892+0000 7f853be00700 1 -- 172.21.15.86:0/667281506 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c004a30 con 0x7f8544101f30 2024-03-22T19:15:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.892+0000 7f853be00700 1 -- 172.21.15.86:0/667281506 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c0054e0 con 0x7f8544101f30 2024-03-22T19:15:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.892+0000 7f853be00700 1 -- 172.21.15.86:0/667281506 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00fb90 con 0x7f8544101f30 2024-03-22T19:15:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.893+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85441add70 con 0x7f8544101f30 2024-03-22T19:15:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.893+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85441ae1d0 con 0x7f8544101f30 2024-03-22T19:15:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.899+0000 7f853be00700 1 -- 172.21.15.86:0/667281506 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f852c004b90 con 0x7f8544101f30 2024-03-22T19:15:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.900+0000 7f855d6a6b80 1 Processor -- start 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.902+0000 7f855d6a6b80 1 -- start start 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d488d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d488de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488e420 0x5604d488e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d47e5990 con 0x5604d488e420 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d47e3350 con 0x5604d488d580 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d47dec40 con 0x5604d488cc20 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f8543400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d488d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f8543400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d488d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54948/0 (socket says 172.21.15.86:54948) 2024-03-22T19:15:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.903+0000 7f8543400700 1 -- 172.21.15.86:0/974464627 learned_addr learned my addr 172.21.15.86:0/974464627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.904+0000 7f8543400700 1 -- 172.21.15.86:0/974464627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 msgr2=0x5604d488de40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.904+0000 7f8543400700 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d488de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.904+0000 7f8543400700 1 -- 172.21.15.86:0/974464627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488e420 msgr2=0x5604d488e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.904+0000 7f8543400700 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488e420 0x5604d488e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.904+0000 7f8543400700 1 -- 172.21.15.86:0/974464627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604d46603b0 con 0x5604d488cc20 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.904+0000 7f8543400700 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d488d040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f853c007d40 tx=0x7f853c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7c201b4050bfea server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.905+0000 7f8533e00700 1 -- 172.21.15.86:0/974464627 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c007e90 con 0x5604d488cc20 2024-03-22T19:15:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.905+0000 7f8533e00700 1 -- 172.21.15.86:0/974464627 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f853c015070 con 0x5604d488cc20 2024-03-22T19:15:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.905+0000 7f8533e00700 1 -- 172.21.15.86:0/974464627 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c019730 con 0x5604d488cc20 2024-03-22T19:15:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.906+0000 7f855d6a6b80 1 -- 172.21.15.86:0/974464627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 msgr2=0x5604d488d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.906+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d488d040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f853c007d40 tx=0x7f853c0106b0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.907+0000 7f855d6a6b80 1 -- 172.21.15.86:0/974464627 shutdown_connections 2024-03-22T19:15:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.907+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488e420 0x5604d488e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.907+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d488d040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.907+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/974464627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d488de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.907+0000 7f855d6a6b80 1 -- 172.21.15.86:0/974464627 >> 172.21.15.86:0/974464627 conn(0x5604d4802220 msgr2=0x5604d4887bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.907+0000 7f853be00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85280a0b10 0x7f85280a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.908+0000 7f853be00700 1 -- 172.21.15.86:0/667281506 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f852c0cb750 con 0x7f8544101f30 2024-03-22T19:15:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.908+0000 7f8542a00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85280a0b10 0x7f85280a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.909+0000 7f8542a00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85280a0b10 0x7f85280a2fd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f853400b820 tx=0x7f853400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.911+0000 7f855d6a6b80 1 -- 172.21.15.86:0/974464627 shutdown_connections 2024-03-22T19:15:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.911+0000 7f855d6a6b80 1 -- 172.21.15.86:0/974464627 wait complete. 2024-03-22T19:15:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.915+0000 7f855d6a6b80 1 -- start start 2024-03-22T19:15:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.915+0000 7f855d6a6b80 1 Processor -- start 2024-03-22T19:15:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d4896b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d4897070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d489e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d477ad80 con 0x5604d488cc20 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d477aa20 con 0x5604d488d580 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d46626a0 con 0x5604d488e420 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.916+0000 7f855d6a6b80 10 client.0 _ll_register_callbacks cb 0x5604d45e63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d4897070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d4897070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58476/0 (socket says 172.21.15.86:58476) 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 -- 172.21.15.86:0/349525770 learned_addr learned my addr 172.21.15.86:0/349525770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 -- 172.21.15.86:0/349525770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 msgr2=0x5604d489e0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d489e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 -- 172.21.15.86:0/349525770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 msgr2=0x5604d4896b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d4896b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.917+0000 7f853a000700 1 -- 172.21.15.86:0/349525770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c007970 con 0x5604d488d580 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.919+0000 7f853a000700 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d4897070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f852c098e00 tx=0x7f852c099620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a46229a7bfc735b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.919+0000 7f8530c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.919+0000 7f8530c00700 1 -- 172.21.15.86:0/349525770 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a12f0 con 0x5604d488d580 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.919+0000 7f8530c00700 1 -- 172.21.15.86:0/349525770 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f852c00fcf0 con 0x5604d488d580 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.919+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604d487fad0 con 0x5604d488d580 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.920+0000 7f8530c00700 1 -- 172.21.15.86:0/349525770 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852c0a12f0 con 0x5604d488d580 2024-03-22T19:15:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.920+0000 7f855d6a6b80 10 client.26225 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.920+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5604d460cb60 con 0x5604d488d580 2024-03-22T19:15:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.921+0000 7f8530c00700 1 -- 172.21.15.86:0/349525770 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c00feb0 con 0x5604d488d580 2024-03-22T19:15:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.921+0000 7f8530c00700 1 -- 172.21.15.86:0/349525770 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f852c098360 con 0x5604d488d580 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f855d6a6b80 10 client.26225 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f855d6a6b80 10 client.26225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f855d6a6b80 20 client.26225 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48a2750 0x5604d48a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f855d6a6b80 4 client.26225 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604d487fad0 con 0x5604d48a2750 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.922+0000 7f8542a00700 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48a2750 0x5604d48a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.923+0000 7f8530c00700 10 client.26225 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.923+0000 7f8542a00700 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48a2750 0x5604d48a4b80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5a69ea96564e14 server_cookie=e1a28dc7678d8a1a in_seq=0 out_seq=0 2024-03-22T19:15:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.924+0000 7f8530c00700 1 -- 172.21.15.86:0/349525770 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5604d487fad0 con 0x5604d48a2750 2024-03-22T19:15:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.924+0000 7f8530c00700 10 client.26225 handle_command_reply: tid=0 2024-03-22T19:15:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.924+0000 7f855d6a6b80 1 client.26225 shutdown 2024-03-22T19:15:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.924+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 msgr2=0x5604d4897070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.924+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d4897070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f852c098e00 tx=0x7f852c099620 comp rx=0 tx=0).stop 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 shutdown_connections 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48a2750 0x5604d48a4b80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d489e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d4896b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/349525770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d4897070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 >> 172.21.15.86:0/349525770 conn(0x5604d4802220 msgr2=0x5604d4887a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f8530c00700 0 client.26225 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 shutdown_connections 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 1 -- 172.21.15.86:0/349525770 wait complete. 2024-03-22T19:15:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.925+0000 7f855d6a6b80 20 client.26225 trim_cache size 0 max 16384 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 Processor -- start 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 -- start start 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d489c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488d580 0x5604d489e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488e420 0x5604d4896040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d48845e0 con 0x5604d488d580 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4897d20 con 0x5604d488e420 2024-03-22T19:15:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.942+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d48893b0 con 0x5604d488cc20 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8542a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d489c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8542a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d489c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:54962/0 (socket says 172.21.15.86:54962) 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488e420 0x5604d4896040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8542a00700 1 -- 172.21.15.86:0/2793257230 learned_addr learned my addr 172.21.15.86:0/2793257230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f853a000700 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488d580 0x5604d489e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 -- 172.21.15.86:0/2793257230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 msgr2=0x5604d489c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d489c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 -- 172.21.15.86:0/2793257230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488d580 msgr2=0x5604d489e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488d580 0x5604d489e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 -- 172.21.15.86:0/2793257230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604d47a4ea0 con 0x5604d488e420 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8542a00700 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d489c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f853a000700 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488d580 0x5604d489e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.943+0000 7f8543400700 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488e420 0x5604d4896040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f853c000c00 tx=0x7f853c010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834350f3e3e45312 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f8538c00700 1 -- 172.21.15.86:0/2793257230 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c010db0 con 0x5604d488e420 2024-03-22T19:15:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f8538c00700 1 -- 172.21.15.86:0/2793257230 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f853c005510 con 0x5604d488e420 2024-03-22T19:15:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f8538c00700 1 -- 172.21.15.86:0/2793257230 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c011440 con 0x5604d488e420 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2793257230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488e420 msgr2=0x5604d4896040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488e420 0x5604d4896040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f853c000c00 tx=0x7f853c010cc0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2793257230 shutdown_connections 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488e420 0x5604d4896040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488cc20 0x5604d489c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2793257230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488d580 0x5604d489e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2793257230 >> 172.21.15.86:0/2793257230 conn(0x5604d4802220 msgr2=0x5604d4894990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2793257230 shutdown_connections 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.944+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2793257230 wait complete. 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 -- start start 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 Processor -- start 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d48ee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d48f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d48f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d48f3550 con 0x5604d488cc20 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d48f36c0 con 0x5604d488d580 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d48f3880 con 0x5604d488e420 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.945+0000 7f855d6a6b80 10 client.0 _ll_register_callbacks cb 0x7f852c098aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d48f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f8542a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d48ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d48f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:58488/0 (socket says 172.21.15.86:58488) 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 -- 172.21.15.86:0/2106552764 learned_addr learned my addr 172.21.15.86:0/2106552764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f8543400700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d48f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 -- 172.21.15.86:0/2106552764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 msgr2=0x5604d48f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d48f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 -- 172.21.15.86:0/2106552764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 msgr2=0x5604d48ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d48ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 -- 172.21.15.86:0/2106552764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8544109ad0 con 0x5604d488d580 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f8542a00700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d48ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f853a000700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d48f07d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f852c078170 tx=0x7f852c098590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b95ec2ed133c78a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f8530c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f8530c00700 1 -- 172.21.15.86:0/2106552764 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a12f0 con 0x5604d488d580 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f8530c00700 1 -- 172.21.15.86:0/2106552764 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f852c022300 con 0x5604d488d580 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.946+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f853c015070 con 0x5604d488d580 2024-03-22T19:15:35.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f8530c00700 1 -- 172.21.15.86:0/2106552764 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852c0224c0 con 0x5604d488d580 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 10 client.26231 fetch_fsmap learned FSMap version 36 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5604d460cb60 con 0x5604d488d580 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f8530c00700 1 -- 172.21.15.86:0/2106552764 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f853c015070 con 0x5604d488d580 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f8530c00700 1 -- 172.21.15.86:0/2106552764 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f852c094d60 con 0x5604d488d580 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 10 client.26231 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 20 client.26231 populate_metadata read hostname 'smithi086' 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48f4ed0 0x5604d48f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 4 client.26231 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604d47d7700 con 0x5604d48f4ed0 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.947+0000 7f8542a00700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48f4ed0 0x5604d48f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f8542a00700 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48f4ed0 0x5604d48f72f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac7784b103afa0d server_cookie=f1bcc9c37bd5271c in_seq=0 out_seq=0 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f8530c00700 10 client.26231 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f8530c00700 1 -- 172.21.15.86:0/2106552764 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5604d47d7700 con 0x5604d48f4ed0 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f8530c00700 10 client.26231 handle_command_reply: tid=0 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 client.26231 shutdown 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 msgr2=0x5604d48f07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d48f07d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f852c078170 tx=0x7f852c098590 comp rx=0 tx=0).stop 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 shutdown_connections 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5604d48f4ed0 0x5604d48f72f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5604d488e420 0x5604d48f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f8530c00700 0 client.26231 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d488cc20 0x5604d48ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 --2- 172.21.15.86:0/2106552764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604d488d580 0x5604d48f07d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 >> 172.21.15.86:0/2106552764 conn(0x5604d4802220 msgr2=0x5604d48913b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.948+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 shutdown_connections 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f855d6a6b80 1 -- 172.21.15.86:0/2106552764 wait complete. 2024-03-22T19:15:35.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f855d6a6b80 20 client.26231 trim_cache size 0 max 16384 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (98 inodes in the stack)", 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "c744d376-abbf-46e1-aef8-f02b44a2f81f": { 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:15:35.956 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85280a0b10 msgr2=0x7f85280a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85280a0b10 0x7f85280a2fd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f853400b820 tx=0x7f853400b040 comp rx=0 tx=0).stop 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544101f30 msgr2=0x7f8544110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544101f30 0x7f8544110500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f852c00d750 tx=0x7f852c00da60 comp rx=0 tx=0).stop 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 shutdown_connections 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f85280a0b10 0x7f85280a2fd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85441028d0 0x7f85441ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8544100d40 0x7f854410ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 --2- 172.21.15.86:0/667281506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8544101f30 0x7f8544110500 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:15:35.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.949+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 >> 172.21.15.86:0/667281506 conn(0x7f85440fc2f0 msgr2=0x7f854410d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:15:35.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.950+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 shutdown_connections 2024-03-22T19:15:35.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:15:35.950+0000 7f8549c00700 1 -- 172.21.15.86:0/667281506 wait complete. 2024-03-22T19:15:35.969 INFO:tasks.cephfs.filesystem:scrub status for tag:c744d376-abbf-46e1-aef8-f02b44a2f81f - {'path': '/', 'tag': 'c744d376-abbf-46e1-aef8-f02b44a2f81f', 'options': 'recursive,force'} 2024-03-22T19:15:36.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:36 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:15:36.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:36 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:36.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:36 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:15:36.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:36 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:15:36.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:36 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:15:36.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:36 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:15:36.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:36 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:15:36.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:36 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:36.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:36 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:15:36.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:36 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:15:36.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:36 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:15:36.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:36 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:15:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:36 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:15:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:36 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:36 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:15:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:36 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:15:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:36 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:15:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:36 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:15:37.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:37 smithi086 ceph-mon[38032]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 83 MiB/s wr, 12.17k op/s 2024-03-22T19:15:37.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:37 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:37.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:37 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:37.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:37 smithi167 ceph-mon[47728]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 83 MiB/s wr, 12.17k op/s 2024-03-22T19:15:37.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:37 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:37.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:37 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:37 smithi131 ceph-mon[78559]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 83 MiB/s wr, 12.17k op/s 2024-03-22T19:15:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:37 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:37.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:37 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:39.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:39 smithi086 ceph-mon[38032]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 77 MiB/s wr, 11.52k op/s 2024-03-22T19:15:39.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:39 smithi167 ceph-mon[47728]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 77 MiB/s wr, 11.52k op/s 2024-03-22T19:15:39.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:39 smithi131 ceph-mon[78559]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 77 MiB/s wr, 11.52k op/s 2024-03-22T19:15:41.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:41 smithi086 ceph-mon[38032]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 11.23k op/s 2024-03-22T19:15:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:41 smithi167 ceph-mon[47728]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 11.23k op/s 2024-03-22T19:15:41.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:41 smithi131 ceph-mon[78559]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 11.23k op/s 2024-03-22T19:15:43.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:43 smithi167 ceph-mon[47728]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 81 MiB/s wr, 12.37k op/s 2024-03-22T19:15:43.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:43 smithi131 ceph-mon[78559]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 81 MiB/s wr, 12.37k op/s 2024-03-22T19:15:44.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:43 smithi086 ceph-mon[38032]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 81 MiB/s wr, 12.37k op/s 2024-03-22T19:15:44.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:44 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:44.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:44 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:45.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:44 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:15:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:45 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:15:45.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:15:45.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:15:45.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:15:45.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:15:45.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:15:45.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:15:45.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:45 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:15:46.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:45 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:15:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:47 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:47.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:47 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:47 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:15:49.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:49 smithi131 ceph-mon[78559]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 90 MiB/s wr, 13.65k op/s 2024-03-22T19:15:49.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:49 smithi131 ceph-mon[78559]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 86 MiB/s wr, 13.05k op/s 2024-03-22T19:15:49.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:49 smithi086 ceph-mon[38032]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 90 MiB/s wr, 13.65k op/s 2024-03-22T19:15:49.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:49 smithi086 ceph-mon[38032]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 86 MiB/s wr, 13.05k op/s 2024-03-22T19:15:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:49 smithi167 ceph-mon[47728]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 90 MiB/s wr, 13.65k op/s 2024-03-22T19:15:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:49 smithi167 ceph-mon[47728]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 86 MiB/s wr, 13.05k op/s 2024-03-22T19:15:51.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:51 smithi131 ceph-mon[78559]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 88 MiB/s wr, 13.48k op/s 2024-03-22T19:15:51.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:51 smithi086 ceph-mon[38032]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 88 MiB/s wr, 13.48k op/s 2024-03-22T19:15:51.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:51 smithi167 ceph-mon[47728]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 88 MiB/s wr, 13.48k op/s 2024-03-22T19:15:52.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:52 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:52.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:52 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:52.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:52 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:53.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:53 smithi131 ceph-mon[78559]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 MiB/s wr, 13.66k op/s 2024-03-22T19:15:53.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:53 smithi086 ceph-mon[38032]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 MiB/s wr, 13.66k op/s 2024-03-22T19:15:53.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:53 smithi167 ceph-mon[47728]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 MiB/s wr, 13.66k op/s 2024-03-22T19:15:55.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:55 smithi086 ceph-mon[38032]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 96 MiB/s wr, 14.66k op/s 2024-03-22T19:15:55.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:55 smithi167 ceph-mon[47728]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 96 MiB/s wr, 14.66k op/s 2024-03-22T19:15:55.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:55 smithi131 ceph-mon[78559]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 96 MiB/s wr, 14.66k op/s 2024-03-22T19:15:57.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:57 smithi086 ceph-mon[38032]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 82 MiB/s wr, 12.71k op/s 2024-03-22T19:15:57.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:57 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:57 smithi167 ceph-mon[47728]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 82 MiB/s wr, 12.71k op/s 2024-03-22T19:15:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:57 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:57.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:57 smithi131 ceph-mon[78559]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 82 MiB/s wr, 12.71k op/s 2024-03-22T19:15:57.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:57 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:15:59.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:15:59 smithi086 ceph-mon[38032]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 85 MiB/s wr, 13.30k op/s 2024-03-22T19:15:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:15:59 smithi167 ceph-mon[47728]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 85 MiB/s wr, 13.30k op/s 2024-03-22T19:15:59.979 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:15:59 smithi131 ceph-mon[78559]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 85 MiB/s wr, 13.30k op/s 2024-03-22T19:16:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:01 smithi167 ceph-mon[47728]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 87 MiB/s wr, 13.73k op/s 2024-03-22T19:16:01.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:01 smithi131 ceph-mon[78559]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 87 MiB/s wr, 13.73k op/s 2024-03-22T19:16:02.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:01 smithi086 ceph-mon[38032]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 87 MiB/s wr, 13.73k op/s 2024-03-22T19:16:02.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:02 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:16:02.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:02 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:16:03.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:02 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:16:03.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:03 smithi167 ceph-mon[47728]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 84 MiB/s wr, 13.06k op/s 2024-03-22T19:16:03.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:03 smithi131 ceph-mon[78559]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 84 MiB/s wr, 13.06k op/s 2024-03-22T19:16:04.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:03 smithi086 ceph-mon[38032]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 84 MiB/s wr, 13.06k op/s 2024-03-22T19:16:05.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:16:05.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:05 smithi131 ceph-mon[78559]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 13.70k op/s 2024-03-22T19:16:06.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:05 smithi086 ceph-mon[38032]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 13.70k op/s 2024-03-22T19:16:06.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:05 smithi167 ceph-mon[47728]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 13.70k op/s 2024-03-22T19:16:07.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:07 smithi167 ceph-mon[47728]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 83 MiB/s wr, 12.32k op/s 2024-03-22T19:16:07.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:07 smithi131 ceph-mon[78559]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 83 MiB/s wr, 12.32k op/s 2024-03-22T19:16:07.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:07 smithi086 ceph-mon[38032]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 83 MiB/s wr, 12.32k op/s 2024-03-22T19:16:09.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:09 smithi086 ceph-mon[38032]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 95 MiB/s wr, 13.96k op/s 2024-03-22T19:16:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:09 smithi131 ceph-mon[78559]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 95 MiB/s wr, 13.96k op/s 2024-03-22T19:16:09.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:09 smithi167 ceph-mon[47728]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 95 MiB/s wr, 13.96k op/s 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 -- 172.21.15.86:0/4228430520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 msgr2=0x7f91ac1043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 --2- 172.21.15.86:0/4228430520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac1043a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009990 tx=0x7f91a4009ca0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 -- 172.21.15.86:0/4228430520 shutdown_connections 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 --2- 172.21.15.86:0/4228430520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 0x7f91ac104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 --2- 172.21.15.86:0/4228430520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 --2- 172.21.15.86:0/4228430520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac1043a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 -- 172.21.15.86:0/4228430520 >> 172.21.15.86:0/4228430520 conn(0x7f91ac0fb1a0 msgr2=0x7f91ac0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 -- 172.21.15.86:0/4228430520 shutdown_connections 2024-03-22T19:16:10.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.787+0000 7f91b2600700 1 -- 172.21.15.86:0/4228430520 wait complete. 2024-03-22T19:16:10.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.789+0000 7f91b2600700 1 Processor -- start 2024-03-22T19:16:10.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 -- start start 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 0x7f91ac1a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac1a4de0 con 0x7f91ac101f10 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac105bc0 con 0x7f91ac1048e0 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.790+0000 7f91b2600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac105d50 con 0x7f91ac0ff5a0 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.791+0000 7f91abe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 0x7f91ac1a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.791+0000 7f91abe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 0x7f91ac1a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:47052/0 (socket says 172.21.15.86:47052) 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.791+0000 7f91abe00700 1 -- 172.21.15.86:0/2527794255 learned_addr learned my addr 172.21.15.86:0/2527794255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.791+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.792+0000 7f91ab400700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.792+0000 7f91aaa00700 1 -- 172.21.15.86:0/2527794255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ac0ff5a0 msgr2=0x7f91ac19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.792+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac19d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.792+0000 7f91aaa00700 1 -- 172.21.15.86:0/2527794255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 msgr2=0x7f91ac1a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.792+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 0x7f91ac1a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.793+0000 7f91aaa00700 1 -- 172.21.15.86:0/2527794255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a40095f0 con 0x7f91ac101f10 2024-03-22T19:16:10.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.794+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac19d7f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f91a400b680 tx=0x7f91a4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.798+0000 7f91a3e00700 1 -- 172.21.15.86:0/2527794255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a401d070 con 0x7f91ac101f10 2024-03-22T19:16:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.798+0000 7f91a3e00700 1 -- 172.21.15.86:0/2527794255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a400be60 con 0x7f91ac101f10 2024-03-22T19:16:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.798+0000 7f91a3e00700 1 -- 172.21.15.86:0/2527794255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40173f0 con 0x7f91ac101f10 2024-03-22T19:16:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.800+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac105fd0 con 0x7f91ac101f10 2024-03-22T19:16:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.800+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac106400 con 0x7f91ac101f10 2024-03-22T19:16:10.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.809+0000 7f91a3e00700 1 -- 172.21.15.86:0/2527794255 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f91a400fc40 con 0x7f91ac101f10 2024-03-22T19:16:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.829+0000 7f91c619fb80 1 Processor -- start 2024-03-22T19:16:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.830+0000 7f91c619fb80 1 -- start start 2024-03-22T19:16:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.832+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.832+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a2ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.832+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a2f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.832+0000 7f91c619fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1986990 con 0x5595a1a2f420 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.832+0000 7f91c619fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1984350 con 0x5595a1a2e580 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.832+0000 7f91c619fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a197fc40 con 0x5595a1a2dc20 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.833+0000 7f91a3e00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f91900a0c10 0x7f91900a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.833+0000 7f91a3e00700 1 -- 172.21.15.86:0/2527794255 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f91a40ccdb0 con 0x7f91ac101f10 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.833+0000 7f91aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a2f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.833+0000 7f91aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a2f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38558/0 (socket says 172.21.15.86:38558) 2024-03-22T19:16:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.833+0000 7f91aaa00700 1 -- 172.21.15.86:0/2075390913 learned_addr learned my addr 172.21.15.86:0/2075390913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91ab400700 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a2ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91abe00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f91900a0c10 0x7f91900a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91aaa00700 1 -- 172.21.15.86:0/2075390913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 msgr2=0x5595a1a2e040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:16:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a2e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91aaa00700 1 -- 172.21.15.86:0/2075390913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 msgr2=0x5595a1a2ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a2ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.834+0000 7f91aaa00700 1 -- 172.21.15.86:0/2075390913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595a18013b0 con 0x5595a1a2f420 2024-03-22T19:16:10.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.835+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a2f8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f91a4000c00 tx=0x7f91a4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1302769bc168723f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:10.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.835+0000 7f91a0c00700 1 -- 172.21.15.86:0/2075390913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a409a560 con 0x5595a1a2f420 2024-03-22T19:16:10.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.836+0000 7f91abe00700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f91900a0c10 0x7f91900a30d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9194009de0 tx=0x7f9194009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:10.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.836+0000 7f91a0c00700 1 -- 172.21.15.86:0/2075390913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a409e390 con 0x5595a1a2f420 2024-03-22T19:16:10.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.837+0000 7f91c619fb80 1 -- 172.21.15.86:0/2075390913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 msgr2=0x5595a1a2f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.837+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a2f8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f91a4000c00 tx=0x7f91a4009ca0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.838+0000 7f91c619fb80 1 -- 172.21.15.86:0/2075390913 shutdown_connections 2024-03-22T19:16:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.838+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a2f8a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.838+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.838+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2075390913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a2ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.838+0000 7f91c619fb80 1 -- 172.21.15.86:0/2075390913 >> 172.21.15.86:0/2075390913 conn(0x5595a19a3220 msgr2=0x5595a1a28bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.839+0000 7f91c619fb80 1 -- 172.21.15.86:0/2075390913 shutdown_connections 2024-03-22T19:16:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.839+0000 7f91c619fb80 1 -- 172.21.15.86:0/2075390913 wait complete. 2024-03-22T19:16:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.840+0000 7f91c619fb80 1 -- start start 2024-03-22T19:16:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.840+0000 7f91c619fb80 1 Processor -- start 2024-03-22T19:16:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.841+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2dc20 0x5595a1a37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.841+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2e580 0x5595a1a37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.841+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.842+0000 7f91aaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2e580 0x5595a1a37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.842+0000 7f91aaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2e580 0x5595a1a37c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:60764/0 (socket says 172.21.15.86:60764) 2024-03-22T19:16:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.842+0000 7f91abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.843+0000 7f91aaa00700 1 -- 172.21.15.86:0/2022193578 learned_addr learned my addr 172.21.15.86:0/2022193578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.843+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a191bd80 con 0x5595a1a2f420 2024-03-22T19:16:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.843+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a191ba20 con 0x5595a1a2dc20 2024-03-22T19:16:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.843+0000 7f91ab400700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2dc20 0x5595a1a37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.843+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a18036a0 con 0x5595a1a2e580 2024-03-22T19:16:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.844+0000 7f91c619fb80 10 client.0 _ll_register_callbacks cb 0x5595a17873a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:16:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.845+0000 7f91abe00700 1 -- 172.21.15.86:0/2022193578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2e580 msgr2=0x5595a1a37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.845+0000 7f91abe00700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2e580 0x5595a1a37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.845+0000 7f91abe00700 1 -- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2dc20 msgr2=0x5595a1a37740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.845+0000 7f91abe00700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2dc20 0x5595a1a37740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.845+0000 7f91abe00700 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac106400 con 0x5595a1a2f420 2024-03-22T19:16:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.845+0000 7f91ab400700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2dc20 0x5595a1a37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:16:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.846+0000 7f91abe00700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3ecc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91940098a0 tx=0x7f91940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c289e1f4ca8e6ad server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.846+0000 7f9199600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:16:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.847+0000 7f9199600700 1 -- 172.21.15.86:0/2022193578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919401d820 con 0x5595a1a2f420 2024-03-22T19:16:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.847+0000 7f9199600700 1 -- 172.21.15.86:0/2022193578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9194023070 con 0x5595a1a2f420 2024-03-22T19:16:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.847+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595a1a20ad0 con 0x5595a1a2f420 2024-03-22T19:16:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.847+0000 7f9199600700 1 -- 172.21.15.86:0/2022193578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919401ce80 con 0x5595a1a2f420 2024-03-22T19:16:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.848+0000 7f91c619fb80 10 client.16113 fetch_fsmap learned FSMap version 40 2024-03-22T19:16:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.848+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5595a17adb60 con 0x5595a1a2f420 2024-03-22T19:16:10.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f9199600700 1 -- 172.21.15.86:0/2022193578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595a1a20ad0 con 0x5595a1a2f420 2024-03-22T19:16:10.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f9199600700 1 -- 172.21.15.86:0/2022193578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9194028b40 con 0x5595a1a2f420 2024-03-22T19:16:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f91c619fb80 10 client.16113 fetch_fsmap finished waiting for FSMap version 40 2024-03-22T19:16:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f91c619fb80 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:16:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f91c619fb80 20 client.16113 populate_metadata read hostname 'smithi086' 2024-03-22T19:16:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f91c619fb80 4 client.16113 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:16:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.850+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595a1978700 con 0x5595a1a43590 2024-03-22T19:16:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.856+0000 7f91ab400700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.860+0000 7f91ab400700 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a459b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfacad6ec4b0580 server_cookie=fa605714ac6cbe08 in_seq=0 out_seq=0 2024-03-22T19:16:10.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.860+0000 7f9199600700 10 client.16113 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.861+0000 7f9199600700 1 -- 172.21.15.86:0/2022193578 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5595a1978700 con 0x5595a1a43590 2024-03-22T19:16:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.861+0000 7f9199600700 10 client.16113 handle_command_reply: tid=0 2024-03-22T19:16:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.861+0000 7f91c619fb80 1 client.16113 shutdown 2024-03-22T19:16:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.861+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 msgr2=0x5595a1a3ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.861+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3ecc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91940098a0 tx=0x7f91940047c0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 shutdown_connections 2024-03-22T19:16:10.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a459b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f9199600700 0 client.16113 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:10.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3ecc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2dc20 0x5595a1a37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2022193578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2e580 0x5595a1a37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.862+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 >> 172.21.15.86:0/2022193578 conn(0x5595a19a3220 msgr2=0x5595a1a281f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.864+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 shutdown_connections 2024-03-22T19:16:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.864+0000 7f91c619fb80 1 -- 172.21.15.86:0/2022193578 wait complete. 2024-03-22T19:16:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.865+0000 7f91c619fb80 20 client.16113 trim_cache size 0 max 16384 2024-03-22T19:16:10.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.911+0000 7f91c619fb80 1 Processor -- start 2024-03-22T19:16:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.911+0000 7f91c619fb80 1 -- start start 2024-03-22T19:16:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.911+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.911+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.912+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.912+0000 7f91c619fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1a2f290 con 0x5595a1a2f420 2024-03-22T19:16:10.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.912+0000 7f91c619fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1a23210 con 0x5595a1a2e580 2024-03-22T19:16:10.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.912+0000 7f91c619fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1a255e0 con 0x5595a1a2dc20 2024-03-22T19:16:10.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.912+0000 7f91abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.913+0000 7f91abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38574/0 (socket says 172.21.15.86:38574) 2024-03-22T19:16:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.913+0000 7f91abe00700 1 -- 172.21.15.86:0/3771367406 learned_addr learned my addr 172.21.15.86:0/3771367406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.913+0000 7f91ab400700 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.914+0000 7f91ab400700 1 -- 172.21.15.86:0/3771367406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 msgr2=0x5595a1a3cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.914+0000 7f91ab400700 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.914+0000 7f91ab400700 1 -- 172.21.15.86:0/3771367406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 msgr2=0x5595a1a3f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.914+0000 7f91ab400700 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.914+0000 7f91ab400700 1 -- 172.21.15.86:0/3771367406 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595a1945ea0 con 0x5595a1a2dc20 2024-03-22T19:16:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.914+0000 7f91abe00700 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:16:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.915+0000 7f91ab400700 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a4a2d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5595a1a46220 tx=0x7f919c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e4f63a642f31d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.915+0000 7f91a1600700 1 -- 172.21.15.86:0/3771367406 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c021070 con 0x5595a1a2dc20 2024-03-22T19:16:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.915+0000 7f91a1600700 1 -- 172.21.15.86:0/3771367406 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919c00f070 con 0x5595a1a2dc20 2024-03-22T19:16:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.915+0000 7f91a1600700 1 -- 172.21.15.86:0/3771367406 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c01c070 con 0x5595a1a2dc20 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 -- 172.21.15.86:0/3771367406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 msgr2=0x5595a1a4a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a4a2d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5595a1a46220 tx=0x7f919c004ad0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 -- 172.21.15.86:0/3771367406 shutdown_connections 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2f420 0x5595a1a3f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a4a2d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 --2- 172.21.15.86:0/3771367406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2e580 0x5595a1a3cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.917+0000 7f91c619fb80 1 -- 172.21.15.86:0/3771367406 >> 172.21.15.86:0/3771367406 conn(0x5595a19a3220 msgr2=0x5595a1a36c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:10.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.918+0000 7f91c619fb80 1 -- 172.21.15.86:0/3771367406 shutdown_connections 2024-03-22T19:16:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.918+0000 7f91c619fb80 1 -- 172.21.15.86:0/3771367406 wait complete. 2024-03-22T19:16:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.919+0000 7f91c619fb80 1 -- start start 2024-03-22T19:16:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.919+0000 7f91c619fb80 1 Processor -- start 2024-03-22T19:16:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2e580 0x5595a1a3aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2f420 0x5595a1a3beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1a3c3f0 con 0x5595a1a2e580 2024-03-22T19:16:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1a3c560 con 0x5595a1a2f420 2024-03-22T19:16:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a1a3c720 con 0x5595a1a2dc20 2024-03-22T19:16:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91c619fb80 10 client.0 _ll_register_callbacks cb 0x7f919401c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:16:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.921+0000 7f91aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2e580 0x5595a1a3aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2e580 0x5595a1a3aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38584/0 (socket says 172.21.15.86:38584) 2024-03-22T19:16:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91aaa00700 1 -- 172.21.15.86:0/2722927009 learned_addr learned my addr 172.21.15.86:0/2722927009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91ab400700 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91ab400700 1 -- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2f420 msgr2=0x5595a1a3beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91ab400700 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2f420 0x5595a1a3beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91ab400700 1 -- 172.21.15.86:0/2722927009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2e580 msgr2=0x5595a1a3aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91ab400700 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2e580 0x5595a1a3aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.922+0000 7f91ab400700 1 -- 172.21.15.86:0/2722927009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595a17adb60 con 0x5595a1a2dc20 2024-03-22T19:16:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.923+0000 7f91ab400700 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a3a960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f919c000f80 tx=0x7f919c007d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8063028fc724484 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.923+0000 7f9199600700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:16:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.925+0000 7f9199600700 1 -- 172.21.15.86:0/2722927009 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c021040 con 0x5595a1a2dc20 2024-03-22T19:16:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.925+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91a409e390 con 0x5595a1a2dc20 2024-03-22T19:16:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.925+0000 7f9199600700 1 -- 172.21.15.86:0/2722927009 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f919c004d60 con 0x5595a1a2dc20 2024-03-22T19:16:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.925+0000 7f9199600700 1 -- 172.21.15.86:0/2722927009 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919c01c040 con 0x5595a1a2dc20 2024-03-22T19:16:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.925+0000 7f91c619fb80 10 client.26245 fetch_fsmap learned FSMap version 40 2024-03-22T19:16:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.925+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f919c009b00 con 0x5595a1a2dc20 2024-03-22T19:16:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f9199600700 1 -- 172.21.15.86:0/2722927009 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919c034080 con 0x5595a1a2dc20 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f9199600700 1 -- 172.21.15.86:0/2722927009 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f919c017760 con 0x5595a1a2dc20 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f91c619fb80 10 client.26245 fetch_fsmap finished waiting for FSMap version 40 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f91c619fb80 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f91c619fb80 20 client.26245 populate_metadata read hostname 'smithi086' 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a97fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f91c619fb80 4 client.26245 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:16:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.926+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595a17cecf0 con 0x5595a1a43590 2024-03-22T19:16:10.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.927+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a97fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.929+0000 7f91aaa00700 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a97fb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d20a566c014fb4f server_cookie=a750ac9d25a8f532 in_seq=0 out_seq=0 2024-03-22T19:16:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.929+0000 7f9199600700 10 client.26245 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.934+0000 7f9199600700 1 -- 172.21.15.86:0/2722927009 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5595a17cecf0 con 0x5595a1a43590 2024-03-22T19:16:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.935+0000 7f9199600700 10 client.26245 handle_command_reply: tid=0 2024-03-22T19:16:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.935+0000 7f91c619fb80 1 client.26245 shutdown 2024-03-22T19:16:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 msgr2=0x5595a1a3a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a3a960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f919c000f80 tx=0x7f919c007d40 comp rx=0 tx=0).stop 2024-03-22T19:16:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 shutdown_connections 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5595a1a43590 0x5595a1a97fb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595a1a2f420 0x5595a1a3beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595a1a2dc20 0x5595a1a3a960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 --2- 172.21.15.86:0/2722927009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a1a2e580 0x5595a1a3aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f9199600700 0 client.26245 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.936+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 >> 172.21.15.86:0/2722927009 conn(0x5595a19a3220 msgr2=0x5595a1a36c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.937+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 shutdown_connections 2024-03-22T19:16:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.937+0000 7f91c619fb80 1 -- 172.21.15.86:0/2722927009 wait complete. 2024-03-22T19:16:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.938+0000 7f91c619fb80 20 client.26245 trim_cache size 0 max 16384 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: "c744d376-abbf-46e1-aef8-f02b44a2f81f": { 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:16:10.940 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f91900a0c10 msgr2=0x7f91900a30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f91900a0c10 0x7f91900a30d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9194009de0 tx=0x7f9194009260 comp rx=0 tx=0).stop 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 msgr2=0x7f91ac19d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac19d7f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f91a400b680 tx=0x7f91a4005b00 comp rx=0 tx=0).stop 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 shutdown_connections 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f91900a0c10 0x7f91900a30d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1048e0 0x7f91ac1a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 --2- 172.21.15.86:0/2527794255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91ac101f10 0x7f91ac19d7f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.939+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 >> 172.21.15.86:0/2527794255 conn(0x7f91ac0fb1a0 msgr2=0x7f91ac0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.940+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 shutdown_connections 2024-03-22T19:16:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:10.940+0000 7f91b2600700 1 -- 172.21.15.86:0/2527794255 wait complete. 2024-03-22T19:16:10.984 INFO:tasks.cephfs.filesystem:scrub status for tag:c744d376-abbf-46e1-aef8-f02b44a2f81f - {'path': '/', 'tag': 'c744d376-abbf-46e1-aef8-f02b44a2f81f', 'options': 'recursive,force'} 2024-03-22T19:16:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:11 smithi131 ceph-mon[78559]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 13.20k op/s 2024-03-22T19:16:11.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:11 smithi086 ceph-mon[38032]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 13.20k op/s 2024-03-22T19:16:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:11 smithi167 ceph-mon[47728]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 13.20k op/s 2024-03-22T19:16:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:13 smithi131 ceph-mon[78559]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 92 MiB/s wr, 13.34k op/s 2024-03-22T19:16:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:13 smithi086 ceph-mon[38032]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 92 MiB/s wr, 13.34k op/s 2024-03-22T19:16:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:13 smithi167 ceph-mon[47728]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 92 MiB/s wr, 13.34k op/s 2024-03-22T19:16:15.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:15 smithi131 ceph-mon[78559]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 94 MiB/s wr, 13.80k op/s 2024-03-22T19:16:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:15 smithi086 ceph-mon[38032]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 94 MiB/s wr, 13.80k op/s 2024-03-22T19:16:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:15 smithi167 ceph-mon[47728]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 94 MiB/s wr, 13.80k op/s 2024-03-22T19:16:17.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:17 smithi131 ceph-mon[78559]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 13.39k op/s 2024-03-22T19:16:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:17 smithi086 ceph-mon[38032]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 13.39k op/s 2024-03-22T19:16:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:17 smithi167 ceph-mon[47728]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 13.39k op/s 2024-03-22T19:16:19.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:19 smithi131 ceph-mon[78559]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 89 MiB/s wr, 13.31k op/s 2024-03-22T19:16:19.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:19 smithi086 ceph-mon[38032]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 89 MiB/s wr, 13.31k op/s 2024-03-22T19:16:19.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:19 smithi167 ceph-mon[47728]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 89 MiB/s wr, 13.31k op/s 2024-03-22T19:16:21.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:21 smithi131 ceph-mon[78559]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 82 MiB/s wr, 12.36k op/s 2024-03-22T19:16:21.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:21 smithi086 ceph-mon[38032]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 82 MiB/s wr, 12.36k op/s 2024-03-22T19:16:21.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:21 smithi167 ceph-mon[47728]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 82 MiB/s wr, 12.36k op/s 2024-03-22T19:16:23.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:23 smithi131 ceph-mon[78559]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 13.74k op/s 2024-03-22T19:16:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:23 smithi086 ceph-mon[38032]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 13.74k op/s 2024-03-22T19:16:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:23 smithi167 ceph-mon[47728]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 13.74k op/s 2024-03-22T19:16:25.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:25 smithi131 ceph-mon[78559]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 76 MiB/s wr, 11.58k op/s 2024-03-22T19:16:25.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:25 smithi086 ceph-mon[38032]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 76 MiB/s wr, 11.58k op/s 2024-03-22T19:16:25.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:25 smithi167 ceph-mon[47728]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 76 MiB/s wr, 11.58k op/s 2024-03-22T19:16:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:27 smithi167 ceph-mon[47728]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 81 MiB/s wr, 12.56k op/s 2024-03-22T19:16:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:27 smithi131 ceph-mon[78559]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 81 MiB/s wr, 12.56k op/s 2024-03-22T19:16:27.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:27 smithi086 ceph-mon[38032]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 81 MiB/s wr, 12.56k op/s 2024-03-22T19:16:29.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:29 smithi131 ceph-mon[78559]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 MiB/s wr, 13.33k op/s 2024-03-22T19:16:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:29 smithi086 ceph-mon[38032]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 MiB/s wr, 13.33k op/s 2024-03-22T19:16:29.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:29 smithi167 ceph-mon[47728]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 MiB/s wr, 13.33k op/s 2024-03-22T19:16:31.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:31 smithi131 ceph-mon[78559]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 75 MiB/s wr, 11.90k op/s 2024-03-22T19:16:31.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:31 smithi086 ceph-mon[38032]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 75 MiB/s wr, 11.90k op/s 2024-03-22T19:16:31.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:31 smithi167 ceph-mon[47728]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 75 MiB/s wr, 11.90k op/s 2024-03-22T19:16:33.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:33 smithi131 ceph-mon[78559]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 87 MiB/s wr, 13.74k op/s 2024-03-22T19:16:33.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:33 smithi086 ceph-mon[38032]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 87 MiB/s wr, 13.74k op/s 2024-03-22T19:16:33.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:33 smithi167 ceph-mon[47728]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 87 MiB/s wr, 13.74k op/s 2024-03-22T19:16:35.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:35 smithi131 ceph-mon[78559]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 MiB/s wr, 11.23k op/s 2024-03-22T19:16:35.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:35 smithi086 ceph-mon[38032]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 MiB/s wr, 11.23k op/s 2024-03-22T19:16:35.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:35 smithi167 ceph-mon[47728]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 MiB/s wr, 11.23k op/s 2024-03-22T19:16:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:37 smithi131 ceph-mon[78559]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 81 MiB/s wr, 12.49k op/s 2024-03-22T19:16:37.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:37 smithi086 ceph-mon[38032]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 81 MiB/s wr, 12.49k op/s 2024-03-22T19:16:37.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:37 smithi167 ceph-mon[47728]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 81 MiB/s wr, 12.49k op/s 2024-03-22T19:16:39.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:39 smithi131 ceph-mon[78559]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 86 MiB/s wr, 12.99k op/s 2024-03-22T19:16:39.763 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:39 smithi086 ceph-mon[38032]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 86 MiB/s wr, 12.99k op/s 2024-03-22T19:16:39.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:39 smithi167 ceph-mon[47728]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 86 MiB/s wr, 12.99k op/s 2024-03-22T19:16:40.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 -- 172.21.15.86:0/1646188857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c10f0f0 msgr2=0x7fa91c1114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 --2- 172.21.15.86:0/1646188857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c10f0f0 0x7fa91c1114e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b0a0 tx=0x7fa91400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 -- 172.21.15.86:0/1646188857 shutdown_connections 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 --2- 172.21.15.86:0/1646188857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c10f0f0 0x7fa91c1114e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 --2- 172.21.15.86:0/1646188857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c072a00 0x7fa91c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 --2- 172.21.15.86:0/1646188857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c1082f0 0x7fa91c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.613+0000 7fa921e00700 1 -- 172.21.15.86:0/1646188857 >> 172.21.15.86:0/1646188857 conn(0x7fa91c06da20 msgr2=0x7fa91c06fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:41.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.614+0000 7fa921e00700 1 -- 172.21.15.86:0/1646188857 shutdown_connections 2024-03-22T19:16:41.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.615+0000 7fa921e00700 1 -- 172.21.15.86:0/1646188857 wait complete. 2024-03-22T19:16:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.616+0000 7fa921e00700 1 Processor -- start 2024-03-22T19:16:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.616+0000 7fa921e00700 1 -- start start 2024-03-22T19:16:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.617+0000 7fa921e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 0x7fa91c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.617+0000 7fa921e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 0x7fa91c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.617+0000 7fa920a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 0x7fa91c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.617+0000 7fa91be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 0x7fa91c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.617+0000 7fa920a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 0x7fa91c19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:56558/0 (socket says 172.21.15.86:56558) 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.617+0000 7fa91be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 0x7fa91c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:57888/0 (socket says 172.21.15.86:57888) 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.618+0000 7fa921e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0f0 0x7fa91c1a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.618+0000 7fa921e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a70b0 con 0x7fa91c10f0f0 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.618+0000 7fa921e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a71f0 con 0x7fa91c1082f0 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.618+0000 7fa921e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a7380 con 0x7fa91c072a00 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.618+0000 7fa920a00700 1 -- 172.21.15.86:0/2737857211 learned_addr learned my addr 172.21.15.86:0/2737857211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa91be00700 1 -- 172.21.15.86:0/2737857211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 msgr2=0x7fa91c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa921400700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0f0 0x7fa91c1a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa91be00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 0x7fa91c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa91be00700 1 -- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0f0 msgr2=0x7fa91c1a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa91be00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0f0 0x7fa91c1a6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa91be00700 1 -- 172.21.15.86:0/2737857211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914009d00 con 0x7fa91c1082f0 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.619+0000 7fa920a00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 0x7fa91c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:16:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.620+0000 7fa91be00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 0x7fa91c19fac0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa91000b450 tx=0x7fa91000b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.622+0000 7fa919600700 1 -- 172.21.15.86:0/2737857211 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011830 con 0x7fa91c1082f0 2024-03-22T19:16:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.622+0000 7fa919600700 1 -- 172.21.15.86:0/2737857211 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa910011e50 con 0x7fa91c1082f0 2024-03-22T19:16:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.622+0000 7fa919600700 1 -- 172.21.15.86:0/2737857211 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910010de0 con 0x7fa91c1082f0 2024-03-22T19:16:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.623+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c108f20 con 0x7fa91c1082f0 2024-03-22T19:16:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.624+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c109430 con 0x7fa91c1082f0 2024-03-22T19:16:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.637+0000 7fa919600700 1 -- 172.21.15.86:0/2737857211 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa910010480 con 0x7fa91c1082f0 2024-03-22T19:16:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.640+0000 7fa936fb2b80 1 Processor -- start 2024-03-22T19:16:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.640+0000 7fa919600700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa9040a50a0 0x7fa9040a7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.641+0000 7fa919600700 1 -- 172.21.15.86:0/2737857211 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa9100cba30 con 0x7fa91c1082f0 2024-03-22T19:16:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.641+0000 7fa921400700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa9040a50a0 0x7fa9040a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.641+0000 7fa936fb2b80 1 -- start start 2024-03-22T19:16:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.642+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc61a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.642+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc61b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.642+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc622270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.642+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc572990 con 0x5560cc61b9e0 2024-03-22T19:16:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.642+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc570350 con 0x5560cc619de0 2024-03-22T19:16:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.642+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc56bc40 con 0x5560cc61b020 2024-03-22T19:16:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.643+0000 7fa921400700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa9040a50a0 0x7fa9040a7560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b830 tx=0x7fa914006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.643+0000 7fa921400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc622270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.643+0000 7fa921400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc622270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59484/0 (socket says 172.21.15.86:59484) 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.643+0000 7fa921400700 1 -- 172.21.15.86:0/1617887822 learned_addr learned my addr 172.21.15.86:0/1617887822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa921400700 1 -- 172.21.15.86:0/1617887822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 msgr2=0x5560cc61b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa91be00700 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc61b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa921400700 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc61b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa921400700 1 -- 172.21.15.86:0/1617887822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 msgr2=0x5560cc61a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa921400700 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc61a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa921400700 1 -- 172.21.15.86:0/1617887822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560cc3ed3b0 con 0x5560cc61b9e0 2024-03-22T19:16:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.644+0000 7fa91be00700 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc61b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:16:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.645+0000 7fa921400700 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc622270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b0a0 tx=0x7fa91401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621b47485160f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.645+0000 7fa90dc00700 1 -- 172.21.15.86:0/1617887822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914009450 con 0x5560cc61b9e0 2024-03-22T19:16:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.646+0000 7fa90dc00700 1 -- 172.21.15.86:0/1617887822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa914029460 con 0x5560cc61b9e0 2024-03-22T19:16:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.646+0000 7fa90dc00700 1 -- 172.21.15.86:0/1617887822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140052a0 con 0x5560cc61b9e0 2024-03-22T19:16:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 -- 172.21.15.86:0/1617887822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 msgr2=0x5560cc622270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc622270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b0a0 tx=0x7fa91401b000 comp rx=0 tx=0).stop 2024-03-22T19:16:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 -- 172.21.15.86:0/1617887822 shutdown_connections 2024-03-22T19:16:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc622270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc61a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/1617887822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc61b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.647+0000 7fa936fb2b80 1 -- 172.21.15.86:0/1617887822 >> 172.21.15.86:0/1617887822 conn(0x5560cc58f220 msgr2=0x5560cc614e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.648+0000 7fa936fb2b80 1 -- 172.21.15.86:0/1617887822 shutdown_connections 2024-03-22T19:16:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.648+0000 7fa936fb2b80 1 -- 172.21.15.86:0/1617887822 wait complete. 2024-03-22T19:16:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.649+0000 7fa936fb2b80 1 -- start start 2024-03-22T19:16:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.649+0000 7fa936fb2b80 1 Processor -- start 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.650+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc62c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.650+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 0x5560cc62ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.651+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b9e0 0x5560cc633a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.651+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc507d80 con 0x5560cc619de0 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.651+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc507a20 con 0x5560cc61b020 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.651+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc3ef6a0 con 0x5560cc61b9e0 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.651+0000 7fa936fb2b80 10 client.0 _ll_register_callbacks cb 0x5560cc3733a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:16:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.651+0000 7fa920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc62c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 0x5560cc62ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 0x5560cc62ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:57908/0 (socket says 172.21.15.86:57908) 2024-03-22T19:16:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 -- 172.21.15.86:0/3615612423 learned_addr learned my addr 172.21.15.86:0/3615612423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa921400700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b9e0 0x5560cc633a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 -- 172.21.15.86:0/3615612423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b9e0 msgr2=0x5560cc633a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b9e0 0x5560cc633a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 -- 172.21.15.86:0/3615612423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 msgr2=0x5560cc62c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc62c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.652+0000 7fa91be00700 1 -- 172.21.15.86:0/3615612423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c112100 con 0x5560cc61b020 2024-03-22T19:16:41.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.653+0000 7fa921400700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b9e0 0x5560cc633a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:16:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.657+0000 7fa91be00700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 0x5560cc62ca30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa91000be20 tx=0x7fa910009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1138e4ad17a9792 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.657+0000 7fa902000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:16:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.659+0000 7fa902000700 1 -- 172.21.15.86:0/3615612423 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100987a0 con 0x5560cc61b020 2024-03-22T19:16:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.659+0000 7fa902000700 1 -- 172.21.15.86:0/3615612423 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa9100990b0 con 0x5560cc61b020 2024-03-22T19:16:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.659+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560cc60cc30 con 0x5560cc61b020 2024-03-22T19:16:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.660+0000 7fa902000700 1 -- 172.21.15.86:0/3615612423 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91009cee0 con 0x5560cc61b020 2024-03-22T19:16:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.660+0000 7fa936fb2b80 10 client.26249 fetch_fsmap learned FSMap version 40 2024-03-22T19:16:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.660+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5560cc399b60 con 0x5560cc61b020 2024-03-22T19:16:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.660+0000 7fa902000700 1 -- 172.21.15.86:0/3615612423 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560cc60cc30 con 0x5560cc61b020 2024-03-22T19:16:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.660+0000 7fa902000700 1 -- 172.21.15.86:0/3615612423 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9100a10a0 con 0x5560cc61b020 2024-03-22T19:16:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa936fb2b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 40 2024-03-22T19:16:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa936fb2b80 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:16:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa936fb2b80 20 client.26249 populate_metadata read hostname 'smithi086' 2024-03-22T19:16:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc6381c0 0x5560cc63a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa936fb2b80 4 client.26249 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:16:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560cc564700 con 0x5560cc6381c0 2024-03-22T19:16:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.661+0000 7fa920a00700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc6381c0 0x5560cc63a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.666+0000 7fa920a00700 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc6381c0 0x5560cc63a5e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6688c361a359b0 server_cookie=4a5399374392c1bc in_seq=0 out_seq=0 2024-03-22T19:16:41.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.666+0000 7fa902000700 10 client.26249 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:41.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.668+0000 7fa902000700 1 -- 172.21.15.86:0/3615612423 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5560cc564700 con 0x5560cc6381c0 2024-03-22T19:16:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.668+0000 7fa902000700 10 client.26249 handle_command_reply: tid=0 2024-03-22T19:16:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.668+0000 7fa936fb2b80 1 client.26249 shutdown 2024-03-22T19:16:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.668+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 msgr2=0x5560cc62ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.669+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 0x5560cc62ca30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa91000be20 tx=0x7fa910009280 comp rx=0 tx=0).stop 2024-03-22T19:16:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 shutdown_connections 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc6381c0 0x5560cc63a5e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b9e0 0x5560cc633a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc62c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3615612423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b020 0x5560cc62ca30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa902000700 0 client.26249 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.670+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 >> 172.21.15.86:0/3615612423 conn(0x5560cc58f220 msgr2=0x5560cc614350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.671+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 shutdown_connections 2024-03-22T19:16:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.671+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3615612423 wait complete. 2024-03-22T19:16:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.672+0000 7fa936fb2b80 20 client.26249 trim_cache size 0 max 16384 2024-03-22T19:16:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.710+0000 7fa936fb2b80 1 Processor -- start 2024-03-22T19:16:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.712+0000 7fa936fb2b80 1 -- start start 2024-03-22T19:16:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc631a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc634120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b9e0 0x5560cc62b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc62c060 con 0x5560cc619de0 2024-03-22T19:16:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc61a590 con 0x5560cc61b9e0 2024-03-22T19:16:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc61ae10 con 0x5560cc61b020 2024-03-22T19:16:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.713+0000 7fa91be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc634120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:41 smithi131 ceph-mon[78559]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 MiB/s wr, 10.91k op/s 2024-03-22T19:16:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.715+0000 7fa920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc631a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.715+0000 7fa920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc631a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59510/0 (socket says 172.21.15.86:59510) 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.715+0000 7fa920a00700 1 -- 172.21.15.86:0/3845839298 learned_addr learned my addr 172.21.15.86:0/3845839298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.716+0000 7fa920a00700 1 -- 172.21.15.86:0/3845839298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 msgr2=0x5560cc634120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.716+0000 7fa920a00700 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc634120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.716+0000 7fa920a00700 1 -- 172.21.15.86:0/3845839298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b9e0 msgr2=0x5560cc62b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.716+0000 7fa920a00700 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b9e0 0x5560cc62b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.716+0000 7fa920a00700 1 -- 172.21.15.86:0/3845839298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560cc531ea0 con 0x5560cc619de0 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.718+0000 7fa920a00700 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc631a20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa9080092d0 tx=0x7fa9080099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57f393691113b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.719+0000 7fa90e600700 1 -- 172.21.15.86:0/3845839298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908016d70 con 0x5560cc619de0 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.719+0000 7fa90e600700 1 -- 172.21.15.86:0/3845839298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9080148d0 con 0x5560cc619de0 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.719+0000 7fa90e600700 1 -- 172.21.15.86:0/3845839298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908023760 con 0x5560cc619de0 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.721+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3845839298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 msgr2=0x5560cc631a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.721+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc631a20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa9080092d0 tx=0x7fa9080099e0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.724+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3845839298 shutdown_connections 2024-03-22T19:16:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.724+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc61b9e0 0x5560cc62b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.725+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc619de0 0x5560cc631a20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.725+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/3845839298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc634120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.725+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3845839298 >> 172.21.15.86:0/3845839298 conn(0x5560cc58f220 msgr2=0x5560cc621fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.725+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3845839298 shutdown_connections 2024-03-22T19:16:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.725+0000 7fa936fb2b80 1 -- 172.21.15.86:0/3845839298 wait complete. 2024-03-22T19:16:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.725+0000 7fa936fb2b80 1 -- start start 2024-03-22T19:16:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.726+0000 7fa936fb2b80 1 Processor -- start 2024-03-22T19:16:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.726+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc681f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.726+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc682440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa936fb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc689430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc689970 con 0x5560cc61b9e0 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc689ab0 con 0x5560cc619de0 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa936fb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cc689c40 con 0x5560cc61b020 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa936fb2b80 10 client.0 _ll_register_callbacks cb 0x7fa910098ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa920a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc681f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.728+0000 7fa920a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc681f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:57930/0 (socket says 172.21.15.86:57930) 2024-03-22T19:16:41.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.728+0000 7fa920a00700 1 -- 172.21.15.86:0/734393190 learned_addr learned my addr 172.21.15.86:0/734393190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:16:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.727+0000 7fa91be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc682440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.728+0000 7fa921400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc689430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.729+0000 7fa920a00700 1 -- 172.21.15.86:0/734393190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 msgr2=0x5560cc682440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa920a00700 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc682440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa920a00700 1 -- 172.21.15.86:0/734393190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 msgr2=0x5560cc689430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:16:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa920a00700 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc689430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa920a00700 1 -- 172.21.15.86:0/734393190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908009d40 con 0x5560cc619de0 2024-03-22T19:16:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa921400700 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc689430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:16:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa902000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:16:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.731+0000 7fa920a00700 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc681f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa908018010 tx=0x7fa908014410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f4ac8f69075409f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:16:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.732+0000 7fa902000700 1 -- 172.21.15.86:0/734393190 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908014c40 con 0x5560cc619de0 2024-03-22T19:16:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.732+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa914029460 con 0x5560cc619de0 2024-03-22T19:16:41.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.732+0000 7fa902000700 1 -- 172.21.15.86:0/734393190 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa908014da0 con 0x5560cc619de0 2024-03-22T19:16:41.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.733+0000 7fa902000700 1 -- 172.21.15.86:0/734393190 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa908014c40 con 0x5560cc619de0 2024-03-22T19:16:41.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.735+0000 7fa936fb2b80 10 client.26252 fetch_fsmap learned FSMap version 40 2024-03-22T19:16:41.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.735+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5560cc3f1cd0 con 0x5560cc619de0 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.735+0000 7fa902000700 1 -- 172.21.15.86:0/734393190 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa914029460 con 0x5560cc619de0 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa902000700 1 -- 172.21.15.86:0/734393190 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa908011050 con 0x5560cc619de0 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa936fb2b80 10 client.26252 fetch_fsmap finished waiting for FSMap version 40 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa936fb2b80 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa936fb2b80 20 client.26252 populate_metadata read hostname 'smithi086' 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc68b600 0x5560cc68da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa936fb2b80 4 client.26252 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:16:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.736+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560cc3bacf0 con 0x5560cc68b600 2024-03-22T19:16:41.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.737+0000 7fa91be00700 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc68b600 0x5560cc68da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:16:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.746+0000 7fa902000700 10 client.26252 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.746+0000 7fa91be00700 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc68b600 0x5560cc68da20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35058083245ae88d server_cookie=4bb7881d4db583a5 in_seq=0 out_seq=0 2024-03-22T19:16:41.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.750+0000 7fa902000700 1 -- 172.21.15.86:0/734393190 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5560cc3bacf0 con 0x5560cc68b600 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.750+0000 7fa902000700 10 client.26252 handle_command_reply: tid=0 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.750+0000 7fa936fb2b80 1 client.26252 shutdown 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 msgr2=0x5560cc681f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc681f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa908018010 tx=0x7fa908014410 comp rx=0 tx=0).stop 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 shutdown_connections 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5560cc68b600 0x5560cc68da20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560cc61b9e0 0x5560cc689430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa902000700 0 client.26252 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560cc619de0 0x5560cc681f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 --2- 172.21.15.86:0/734393190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560cc61b020 0x5560cc682440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 >> 172.21.15.86:0/734393190 conn(0x5560cc58f220 msgr2=0x5560cc62b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 shutdown_connections 2024-03-22T19:16:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.751+0000 7fa936fb2b80 1 -- 172.21.15.86:0/734393190 wait complete. 2024-03-22T19:16:41.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.752+0000 7fa936fb2b80 20 client.26252 trim_cache size 0 max 16384 2024-03-22T19:16:41.754 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:16:41.754 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-22T19:16:41.754 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:16:41.754 INFO:teuthology.orchestra.run.smithi086.stdout: "c744d376-abbf-46e1-aef8-f02b44a2f81f": { 2024-03-22T19:16:41.754 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:16:41.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:16:41.755 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:16:41.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:16:41.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:16:41.755 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:16:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa9040a50a0 msgr2=0x7fa9040a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa9040a50a0 0x7fa9040a7560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b830 tx=0x7fa914006000 comp rx=0 tx=0).stop 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 msgr2=0x7fa91c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 0x7fa91c19fac0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa91000b450 tx=0x7fa91000b760 comp rx=0 tx=0).stop 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 shutdown_connections 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa9040a50a0 0x7fa9040a7560 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0f0 0x7fa91c1a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa91c072a00 0x7fa91c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 --2- 172.21.15.86:0/2737857211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa91c1082f0 0x7fa91c19fac0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.755+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 >> 172.21.15.86:0/2737857211 conn(0x7fa91c06da20 msgr2=0x7fa91c10b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:16:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.756+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 shutdown_connections 2024-03-22T19:16:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:16:41.756+0000 7fa921e00700 1 -- 172.21.15.86:0/2737857211 wait complete. 2024-03-22T19:16:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:41 smithi086 ceph-mon[38032]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 MiB/s wr, 10.91k op/s 2024-03-22T19:16:41.805 INFO:tasks.cephfs.filesystem:scrub status for tag:c744d376-abbf-46e1-aef8-f02b44a2f81f - {'path': '/', 'tag': 'c744d376-abbf-46e1-aef8-f02b44a2f81f', 'options': 'recursive,force'} 2024-03-22T19:16:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:41 smithi167 ceph-mon[47728]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 MiB/s wr, 10.91k op/s 2024-03-22T19:16:43.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:43 smithi131 ceph-mon[78559]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 MiB/s wr, 11.14k op/s 2024-03-22T19:16:43.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:43 smithi086 ceph-mon[38032]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 MiB/s wr, 11.14k op/s 2024-03-22T19:16:43.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:43 smithi167 ceph-mon[47728]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 MiB/s wr, 11.14k op/s 2024-03-22T19:16:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:45 smithi131 ceph-mon[78559]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 80 MiB/s wr, 11.69k op/s 2024-03-22T19:16:45.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:45 smithi086 ceph-mon[38032]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 80 MiB/s wr, 11.69k op/s 2024-03-22T19:16:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:45 smithi167 ceph-mon[47728]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 80 MiB/s wr, 11.69k op/s 2024-03-22T19:16:46.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:46 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:16:46.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:46 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:16:46.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:46 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:16:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:47 smithi131 ceph-mon[78559]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 81 MiB/s wr, 11.75k op/s 2024-03-22T19:16:47.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:47 smithi086 ceph-mon[38032]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 81 MiB/s wr, 11.75k op/s 2024-03-22T19:16:47.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:47 smithi167 ceph-mon[47728]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 81 MiB/s wr, 11.75k op/s 2024-03-22T19:16:49.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:49 smithi086 ceph-mon[38032]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 87 MiB/s wr, 12.59k op/s 2024-03-22T19:16:49.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:16:49.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:16:49.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:49 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:16:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:49 smithi167 ceph-mon[47728]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 87 MiB/s wr, 12.59k op/s 2024-03-22T19:16:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:16:49.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:16:49.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:49 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:16:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:49 smithi131 ceph-mon[78559]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 87 MiB/s wr, 12.59k op/s 2024-03-22T19:16:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:16:49.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:16:49.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:49 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:16:51.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:51 smithi086 ceph-mon[38032]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 MiB/s wr, 11.18k op/s 2024-03-22T19:16:51.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:51 smithi167 ceph-mon[47728]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 MiB/s wr, 11.18k op/s 2024-03-22T19:16:51.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:51 smithi131 ceph-mon[78559]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 MiB/s wr, 11.18k op/s 2024-03-22T19:16:53.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:53 smithi167 ceph-mon[47728]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 76 MiB/s wr, 11.04k op/s 2024-03-22T19:16:53.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:53 smithi131 ceph-mon[78559]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 76 MiB/s wr, 11.04k op/s 2024-03-22T19:16:54.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:53 smithi086 ceph-mon[38032]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 76 MiB/s wr, 11.04k op/s 2024-03-22T19:16:55.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:55 smithi167 ceph-mon[47728]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 MiB/s wr, 13.79k op/s 2024-03-22T19:16:55.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:55 smithi131 ceph-mon[78559]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 MiB/s wr, 13.79k op/s 2024-03-22T19:16:56.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:55 smithi086 ceph-mon[38032]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 MiB/s wr, 13.79k op/s 2024-03-22T19:16:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:57 smithi131 ceph-mon[78559]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 79 MiB/s wr, 11.45k op/s 2024-03-22T19:16:57.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:57 smithi086 ceph-mon[38032]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 79 MiB/s wr, 11.45k op/s 2024-03-22T19:16:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:57 smithi167 ceph-mon[47728]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 79 MiB/s wr, 11.45k op/s 2024-03-22T19:16:59.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:16:59 smithi131 ceph-mon[78559]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 88 MiB/s wr, 12.96k op/s 2024-03-22T19:16:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:16:59 smithi086 ceph-mon[38032]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 88 MiB/s wr, 12.96k op/s 2024-03-22T19:16:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:16:59 smithi167 ceph-mon[47728]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 88 MiB/s wr, 12.96k op/s 2024-03-22T19:17:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:01 smithi131 ceph-mon[78559]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 79 MiB/s wr, 11.69k op/s 2024-03-22T19:17:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:01 smithi086 ceph-mon[38032]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 79 MiB/s wr, 11.69k op/s 2024-03-22T19:17:01.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:01 smithi167 ceph-mon[47728]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 79 MiB/s wr, 11.69k op/s 2024-03-22T19:17:03.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:03 smithi131 ceph-mon[78559]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 77 MiB/s wr, 11.32k op/s 2024-03-22T19:17:03.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:03 smithi086 ceph-mon[38032]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 77 MiB/s wr, 11.32k op/s 2024-03-22T19:17:03.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:03 smithi167 ceph-mon[47728]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 77 MiB/s wr, 11.32k op/s 2024-03-22T19:17:05.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:05 smithi086 ceph-mon[38032]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 91 MiB/s wr, 13.33k op/s 2024-03-22T19:17:05.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:05 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:17:05.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:05 smithi167 ceph-mon[47728]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 91 MiB/s wr, 13.33k op/s 2024-03-22T19:17:05.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:05 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:17:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:05 smithi131 ceph-mon[78559]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 91 MiB/s wr, 13.33k op/s 2024-03-22T19:17:05.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:05 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:17:07.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:07 smithi131 ceph-mon[78559]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 78 MiB/s wr, 11.65k op/s 2024-03-22T19:17:07.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:07 smithi086 ceph-mon[38032]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 78 MiB/s wr, 11.65k op/s 2024-03-22T19:17:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:07 smithi167 ceph-mon[47728]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 78 MiB/s wr, 11.65k op/s 2024-03-22T19:17:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:09 smithi131 ceph-mon[78559]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 87 MiB/s wr, 13.19k op/s 2024-03-22T19:17:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:09 smithi086 ceph-mon[38032]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 87 MiB/s wr, 13.19k op/s 2024-03-22T19:17:09.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:09 smithi167 ceph-mon[47728]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 87 MiB/s wr, 13.19k op/s 2024-03-22T19:17:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:11 smithi131 ceph-mon[78559]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 MiB/s wr, 13.17k op/s 2024-03-22T19:17:11.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:11 smithi086 ceph-mon[38032]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 MiB/s wr, 13.17k op/s 2024-03-22T19:17:11.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:17:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:11 smithi167 ceph-mon[47728]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 MiB/s wr, 13.17k op/s 2024-03-22T19:17:13.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.255+0000 7f3fd1600700 1 -- 172.21.15.86:0/4209246687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 msgr2=0x7f3fcc10d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.255+0000 7f3fd1600700 1 --2- 172.21.15.86:0/4209246687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc10d780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009f90 tx=0x7f3fc400b330 comp rx=0 tx=0).stop 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 -- 172.21.15.86:0/4209246687 shutdown_connections 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 --2- 172.21.15.86:0/4209246687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc10d780 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 --2- 172.21.15.86:0/4209246687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc0721e0 0x7f3fcc072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 --2- 172.21.15.86:0/4209246687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc108520 0x7f3fcc1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 -- 172.21.15.86:0/4209246687 >> 172.21.15.86:0/4209246687 conn(0x7f3fcc06dd20 msgr2=0x7f3fcc070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 -- 172.21.15.86:0/4209246687 shutdown_connections 2024-03-22T19:17:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.256+0000 7f3fd1600700 1 -- 172.21.15.86:0/4209246687 wait complete. 2024-03-22T19:17:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.258+0000 7f3fd1600700 1 Processor -- start 2024-03-22T19:17:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.258+0000 7f3fd1600700 1 -- start start 2024-03-22T19:17:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.259+0000 7f3fd1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc0721e0 0x7f3fcc1a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.259+0000 7f3fd1600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc108520 0x7f3fcc1a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.259+0000 7f3fd1600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc1ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc108520 0x7f3fcc1a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc1ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc1ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47112/0 (socket says 172.21.15.86:47112) 2024-03-22T19:17:13.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 -- 172.21.15.86:0/1755251428 learned_addr learned my addr 172.21.15.86:0/1755251428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:13.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc0721e0 0x7f3fcc1a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1ad6e0 con 0x7f3fcc0721e0 2024-03-22T19:17:13.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1ad820 con 0x7f3fcc108520 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1ad9b0 con 0x7f3fcc108ee0 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 -- 172.21.15.86:0/1755251428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc108520 msgr2=0x7f3fcc1a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc108520 0x7f3fcc1a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 -- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc0721e0 msgr2=0x7f3fcc1a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc0721e0 0x7f3fcc1a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcb400700 1 -- 172.21.15.86:0/1755251428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4009c40 con 0x7f3fcc108ee0 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.260+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc0721e0 0x7f3fcc1a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.261+0000 7f3fca000700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc108520 0x7f3fcc1a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:17:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.261+0000 7f3fcb400700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc1ad110 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009f60 tx=0x7f3fc4004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.262+0000 7f3fc3400700 1 -- 172.21.15.86:0/1755251428 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400e040 con 0x7f3fcc108ee0 2024-03-22T19:17:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.262+0000 7f3fc3400700 1 -- 172.21.15.86:0/1755251428 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc400bbf0 con 0x7f3fcc108ee0 2024-03-22T19:17:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.262+0000 7f3fc3400700 1 -- 172.21.15.86:0/1755251428 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40052a0 con 0x7f3fcc108ee0 2024-03-22T19:17:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.262+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc1adc30 con 0x7f3fcc108ee0 2024-03-22T19:17:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.262+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1ae060 con 0x7f3fcc108ee0 2024-03-22T19:17:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.270+0000 7f3fc3400700 1 -- 172.21.15.86:0/1755251428 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f3fc401d470 con 0x7f3fcc108ee0 2024-03-22T19:17:13.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.272+0000 7f3fc3400700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3fb00a0c60 0x7f3fb00a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.272+0000 7f3fc3400700 1 -- 172.21.15.86:0/1755251428 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3fc40cd5f0 con 0x7f3fcc108ee0 2024-03-22T19:17:13.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.274+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3fb00a0c60 0x7f3fb00a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.279+0000 7f3fe5094b80 1 Processor -- start 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3fb00a0c60 0x7f3fb00a3120 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8005fd0 tx=0x7f3fb8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 -- start start 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 0x5611ed2ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cf420 0x5611ed2cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed226990 con 0x5611ed2cf420 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed224350 con 0x5611ed2ce580 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed21fc40 con 0x5611ed2cdc20 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fcaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 0x5611ed2ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fcaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 0x5611ed2ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:47118/0 (socket says 172.21.15.86:47118) 2024-03-22T19:17:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fcaa00700 1 -- 172.21.15.86:0/2056641221 learned_addr learned my addr 172.21.15.86:0/2056641221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.281+0000 7f3fca000700 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.282+0000 7f3fcaa00700 1 -- 172.21.15.86:0/2056641221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 msgr2=0x5611ed2cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.282+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.282+0000 7f3fcaa00700 1 -- 172.21.15.86:0/2056641221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cf420 msgr2=0x5611ed2cf8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.282+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cf420 0x5611ed2cf8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.282+0000 7f3fcaa00700 1 -- 172.21.15.86:0/2056641221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ed0a13b0 con 0x5611ed2cdc20 2024-03-22T19:17:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.282+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 0x5611ed2ce040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8011f80 tx=0x7f3fb8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0993d68d714ef server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.283+0000 7f3fb7400700 1 -- 172.21.15.86:0/2056641221 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8009ea0 con 0x5611ed2cdc20 2024-03-22T19:17:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.283+0000 7f3fb7400700 1 -- 172.21.15.86:0/2056641221 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb8016070 con 0x5611ed2cdc20 2024-03-22T19:17:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.283+0000 7f3fb7400700 1 -- 172.21.15.86:0/2056641221 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb801da20 con 0x5611ed2cdc20 2024-03-22T19:17:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.285+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2056641221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 msgr2=0x5611ed2ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.285+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 0x5611ed2ce040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8011f80 tx=0x7f3fb8014000 comp rx=0 tx=0).stop 2024-03-22T19:17:13.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.286+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2056641221 shutdown_connections 2024-03-22T19:17:13.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.286+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cf420 0x5611ed2cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.286+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cdc20 0x5611ed2ce040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.286+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2056641221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.286+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2056641221 >> 172.21.15.86:0/2056641221 conn(0x5611ed243220 msgr2=0x5611ed2c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:13.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.287+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2056641221 shutdown_connections 2024-03-22T19:17:13.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.287+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2056641221 wait complete. 2024-03-22T19:17:13.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.287+0000 7f3fe5094b80 1 -- start start 2024-03-22T19:17:13.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.288+0000 7f3fe5094b80 1 Processor -- start 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 0x5611ed2d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cf420 0x5611ed2df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed1bbd80 con 0x5611ed2cdc20 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed1bba20 con 0x5611ed2ce580 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed0a36a0 con 0x5611ed2cf420 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.289+0000 7f3fe5094b80 10 client.0 _ll_register_callbacks cb 0x5611ed0273a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.290+0000 7f3fca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.290+0000 7f3fcaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 0x5611ed2d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.290+0000 7f3fcaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 0x5611ed2d7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49788/0 (socket says 172.21.15.86:49788) 2024-03-22T19:17:13.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.290+0000 7f3fcaa00700 1 -- 172.21.15.86:0/1980153643 learned_addr learned my addr 172.21.15.86:0/1980153643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:13.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.290+0000 7f3fcb400700 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cf420 0x5611ed2df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.291+0000 7f3fcaa00700 1 -- 172.21.15.86:0/1980153643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cf420 msgr2=0x5611ed2df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.291+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cf420 0x5611ed2df100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.291+0000 7f3fcaa00700 1 -- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 msgr2=0x5611ed2d80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.291+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2d80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.291+0000 7f3fcaa00700 1 -- 172.21.15.86:0/1980153643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb8011c60 con 0x5611ed2cdc20 2024-03-22T19:17:13.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.293+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 0x5611ed2d7b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fb801df20 tx=0x7f3fb801df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529e6e7a7455fa41 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:13.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.293+0000 7f3fabe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:17:13.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.294+0000 7f3fabe00700 1 -- 172.21.15.86:0/1980153643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb802d070 con 0x5611ed2cdc20 2024-03-22T19:17:13.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.295+0000 7f3fabe00700 1 -- 172.21.15.86:0/1980153643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3fb8018070 con 0x5611ed2cdc20 2024-03-22T19:17:13.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.295+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611ed2c0ad0 con 0x5611ed2cdc20 2024-03-22T19:17:13.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.297+0000 7f3fabe00700 1 -- 172.21.15.86:0/1980153643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb8009ea0 con 0x5611ed2cdc20 2024-03-22T19:17:13.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.297+0000 7f3fe5094b80 10 client.16137 fetch_fsmap learned FSMap version 41 2024-03-22T19:17:13.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.297+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5611ed04db60 con 0x5611ed2cdc20 2024-03-22T19:17:13.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.297+0000 7f3fabe00700 1 -- 172.21.15.86:0/1980153643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611ed2c0ad0 con 0x5611ed2cdc20 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.298+0000 7f3fabe00700 1 -- 172.21.15.86:0/1980153643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3fb803a070 con 0x5611ed2cdc20 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.298+0000 7f3fe5094b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 41 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.299+0000 7f3fe5094b80 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.299+0000 7f3fe5094b80 20 client.16137 populate_metadata read hostname 'smithi086' 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.299+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed2f47f0 0x5611ed2f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.299+0000 7f3fe5094b80 4 client.16137 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:17:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.299+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611ed218700 con 0x5611ed2f47f0 2024-03-22T19:17:13.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.300+0000 7f3fca000700 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed2f47f0 0x5611ed2f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.306+0000 7f3fabe00700 10 client.16137 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:13.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.307+0000 7f3fca000700 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed2f47f0 0x5611ed2f6c10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedc06c6a314496e server_cookie=a56682fd44ac0afb in_seq=0 out_seq=0 2024-03-22T19:17:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.310+0000 7f3fabe00700 1 -- 172.21.15.86:0/1980153643 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5611ed218700 con 0x5611ed2f47f0 2024-03-22T19:17:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.310+0000 7f3fabe00700 10 client.16137 handle_command_reply: tid=0 2024-03-22T19:17:13.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.310+0000 7f3fe5094b80 1 client.16137 shutdown 2024-03-22T19:17:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.312+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 msgr2=0x5611ed2d7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.312+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 0x5611ed2d7b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fb801df20 tx=0x7f3fb801df50 comp rx=0 tx=0).stop 2024-03-22T19:17:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 shutdown_connections 2024-03-22T19:17:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed2f47f0 0x5611ed2f6c10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2cf420 0x5611ed2df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2cdc20 0x5611ed2d7b80 secure :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fb801df20 tx=0x7f3fb801df50 comp rx=0 tx=0).stop 2024-03-22T19:17:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/1980153643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2ce580 0x5611ed2d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fabe00700 0 client.16137 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 >> 172.21.15.86:0/1980153643 conn(0x5611ed243220 msgr2=0x5611ed2c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.313+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 shutdown_connections 2024-03-22T19:17:13.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.314+0000 7f3fe5094b80 1 -- 172.21.15.86:0/1980153643 wait complete. 2024-03-22T19:17:13.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.314+0000 7f3fe5094b80 20 client.16137 trim_cache size 0 max 16384 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.348+0000 7f3fe5094b80 1 Processor -- start 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 -- start start 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed2c0610 con 0x5611ed2ce580 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed2d64d0 con 0x5611ed2cf420 2024-03-22T19:17:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.349+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed2f7270 con 0x5611ed2d70c0 2024-03-22T19:17:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2df0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:55134/0 (socket says 172.21.15.86:55134) 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 -- 172.21.15.86:0/4039528130 learned_addr learned my addr 172.21.15.86:0/4039528130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fcb400700 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 -- 172.21.15.86:0/4039528130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 msgr2=0x5611ed2c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 -- 172.21.15.86:0/4039528130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 msgr2=0x5611ed2cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.350+0000 7f3fca000700 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.351+0000 7f3fca000700 1 -- 172.21.15.86:0/4039528130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ed1e5ea0 con 0x5611ed2cf420 2024-03-22T19:17:13.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.353+0000 7f3fca000700 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2df0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00a550 tx=0x7f3fbc01dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb63ddfc257f451 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:13.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.358+0000 7f3fb7e00700 1 -- 172.21.15.86:0/4039528130 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc017040 con 0x5611ed2cf420 2024-03-22T19:17:13.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.359+0000 7f3fb7e00700 1 -- 172.21.15.86:0/4039528130 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fbc01d410 con 0x5611ed2cf420 2024-03-22T19:17:13.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.359+0000 7f3fb7e00700 1 -- 172.21.15.86:0/4039528130 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0126b0 con 0x5611ed2cf420 2024-03-22T19:17:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 -- 172.21.15.86:0/4039528130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 msgr2=0x5611ed2df0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2df0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00a550 tx=0x7f3fbc01dad0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 -- 172.21.15.86:0/4039528130 shutdown_connections 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/4039528130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2df0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 -- 172.21.15.86:0/4039528130 >> 172.21.15.86:0/4039528130 conn(0x5611ed243220 msgr2=0x5611ed2c4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 -- 172.21.15.86:0/4039528130 shutdown_connections 2024-03-22T19:17:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.360+0000 7f3fe5094b80 1 -- 172.21.15.86:0/4039528130 wait complete. 2024-03-22T19:17:13.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.361+0000 7f3fe5094b80 1 -- start start 2024-03-22T19:17:13.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.362+0000 7f3fe5094b80 1 Processor -- start 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.362+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.362+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.362+0000 7f3fe5094b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.362+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed2df460 con 0x5611ed2ce580 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.362+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed2dc160 con 0x5611ed2cf420 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.363+0000 7f3fe5094b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ed2dc2f0 con 0x5611ed2d70c0 2024-03-22T19:17:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.363+0000 7f3fe5094b80 10 client.0 _ll_register_callbacks cb 0x7f3fb8011e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:17:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.363+0000 7f3fca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.363+0000 7f3fca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2db6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:55144/0 (socket says 172.21.15.86:55144) 2024-03-22T19:17:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.363+0000 7f3fca000700 1 -- 172.21.15.86:0/2495652552 learned_addr learned my addr 172.21.15.86:0/2495652552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.363+0000 7f3fcb400700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.364+0000 7f3fca000700 1 -- 172.21.15.86:0/2495652552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 msgr2=0x5611ed2dbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.364+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.365+0000 7f3fca000700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2dbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.365+0000 7f3fca000700 1 -- 172.21.15.86:0/2495652552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 msgr2=0x5611ed2dec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.365+0000 7f3fca000700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2dec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.365+0000 7f3fca000700 1 -- 172.21.15.86:0/2495652552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc00ae70 con 0x5611ed2cf420 2024-03-22T19:17:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.365+0000 7f3fcb400700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:17:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.367+0000 7f3fca000700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2db6e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0037e0 tx=0x7f3fbc003810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8ee24c0b1c13ed server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.367+0000 7f3fabe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:17:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.368+0000 7f3fabe00700 1 -- 172.21.15.86:0/2495652552 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc02b070 con 0x5611ed2cf420 2024-03-22T19:17:13.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.368+0000 7f3fabe00700 1 -- 172.21.15.86:0/2495652552 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3fbc003b20 con 0x5611ed2cf420 2024-03-22T19:17:13.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.368+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fb8016070 con 0x5611ed2cf420 2024-03-22T19:17:13.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.370+0000 7f3fabe00700 1 -- 172.21.15.86:0/2495652552 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fbc017040 con 0x5611ed2cf420 2024-03-22T19:17:13.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.370+0000 7f3fe5094b80 10 client.26270 fetch_fsmap learned FSMap version 41 2024-03-22T19:17:13.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.370+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5611ed0a5cd0 con 0x5611ed2cf420 2024-03-22T19:17:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.373+0000 7f3fabe00700 1 -- 172.21.15.86:0/2495652552 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb8016070 con 0x5611ed2cf420 2024-03-22T19:17:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.373+0000 7f3fabe00700 1 -- 172.21.15.86:0/2495652552 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 41) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3fbc004990 con 0x5611ed2cf420 2024-03-22T19:17:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fe5094b80 10 client.26270 fetch_fsmap finished waiting for FSMap version 41 2024-03-22T19:17:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fe5094b80 10 client.26270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:17:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fe5094b80 20 client.26270 populate_metadata read hostname 'smithi086' 2024-03-22T19:17:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed347af0 0x5611ed349f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fe5094b80 4 client.26270 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:17:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5611ed06ecf0 con 0x5611ed347af0 2024-03-22T19:17:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.375+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed347af0 0x5611ed349f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:13.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.376+0000 7f3fcaa00700 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed347af0 0x5611ed349f10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f42e74585d7ee5 server_cookie=569af20152a2185e in_seq=0 out_seq=0 2024-03-22T19:17:13.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.376+0000 7f3fabe00700 10 client.26270 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.380+0000 7f3fabe00700 1 -- 172.21.15.86:0/2495652552 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5611ed06ecf0 con 0x5611ed347af0 2024-03-22T19:17:13.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.380+0000 7f3fabe00700 10 client.26270 handle_command_reply: tid=0 2024-03-22T19:17:13.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.380+0000 7f3fe5094b80 1 client.26270 shutdown 2024-03-22T19:17:13.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 msgr2=0x5611ed2db6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2db6e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0037e0 tx=0x7f3fbc003810 comp rx=0 tx=0).stop 2024-03-22T19:17:13.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 shutdown_connections 2024-03-22T19:17:13.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5611ed347af0 0x5611ed349f10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611ed2d70c0 0x5611ed2dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611ed2ce580 0x5611ed2dec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 --2- 172.21.15.86:0/2495652552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611ed2cf420 0x5611ed2db6e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 >> 172.21.15.86:0/2495652552 conn(0x5611ed243220 msgr2=0x5611ed2c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:13.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.382+0000 7f3fabe00700 0 client.26270 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:13.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.383+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 shutdown_connections 2024-03-22T19:17:13.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.383+0000 7f3fe5094b80 1 -- 172.21.15.86:0/2495652552 wait complete. 2024-03-22T19:17:13.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.384+0000 7f3fe5094b80 20 client.26270 trim_cache size 0 max 16384 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "c744d376-abbf-46e1-aef8-f02b44a2f81f": { 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:17:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:17:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:17:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3fb00a0c60 msgr2=0x7f3fb00a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3fb00a0c60 0x7f3fb00a3120 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8005fd0 tx=0x7f3fb8009310 comp rx=0 tx=0).stop 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 msgr2=0x7f3fcc1ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc1ad110 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009f60 tx=0x7f3fc4004c60 comp rx=0 tx=0).stop 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 shutdown_connections 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f3fb00a0c60 0x7f3fb00a3120 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fcc108ee0 0x7f3fcc1ad110 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fcc0721e0 0x7f3fcc1a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 --2- 172.21.15.86:0/1755251428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc108520 0x7f3fcc1a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:13.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.386+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 >> 172.21.15.86:0/1755251428 conn(0x7f3fcc06dd20 msgr2=0x7f3fcc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:13.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.387+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 shutdown_connections 2024-03-22T19:17:13.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:13.387+0000 7f3fd1600700 1 -- 172.21.15.86:0/1755251428 wait complete. 2024-03-22T19:17:13.425 INFO:tasks.cephfs.filesystem:scrub status for tag:c744d376-abbf-46e1-aef8-f02b44a2f81f - {'path': '/', 'tag': 'c744d376-abbf-46e1-aef8-f02b44a2f81f', 'options': 'recursive,force'} 2024-03-22T19:17:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:13 smithi131 ceph-mon[78559]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 87 MiB/s wr, 13.14k op/s 2024-03-22T19:17:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:13 smithi086 ceph-mon[38032]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 87 MiB/s wr, 13.14k op/s 2024-03-22T19:17:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:13 smithi167 ceph-mon[47728]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 87 MiB/s wr, 13.14k op/s 2024-03-22T19:17:15.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:15 smithi131 ceph-mon[78559]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 93 MiB/s wr, 14.00k op/s 2024-03-22T19:17:15.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:15 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:17:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:15 smithi086 ceph-mon[38032]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 93 MiB/s wr, 14.00k op/s 2024-03-22T19:17:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:15 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:17:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:15 smithi167 ceph-mon[47728]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 93 MiB/s wr, 14.00k op/s 2024-03-22T19:17:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:15 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:17:17.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:17 smithi131 ceph-mon[78559]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 84 MiB/s wr, 12.89k op/s 2024-03-22T19:17:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:17 smithi086 ceph-mon[38032]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 84 MiB/s wr, 12.89k op/s 2024-03-22T19:17:17.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:17 smithi167 ceph-mon[47728]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 84 MiB/s wr, 12.89k op/s 2024-03-22T19:17:19.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:19 smithi131 ceph-mon[78559]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 MiB/s wr, 12.79k op/s 2024-03-22T19:17:19.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:19 smithi086 ceph-mon[38032]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 MiB/s wr, 12.79k op/s 2024-03-22T19:17:19.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:19 smithi167 ceph-mon[47728]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 MiB/s wr, 12.79k op/s 2024-03-22T19:17:21.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:21 smithi131 ceph-mon[78559]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 76 MiB/s wr, 11.61k op/s 2024-03-22T19:17:21.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:21 smithi086 ceph-mon[38032]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 76 MiB/s wr, 11.61k op/s 2024-03-22T19:17:21.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:21 smithi167 ceph-mon[47728]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 76 MiB/s wr, 11.61k op/s 2024-03-22T19:17:23.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:23 smithi131 ceph-mon[78559]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 71 MiB/s wr, 10.71k op/s 2024-03-22T19:17:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:23 smithi086 ceph-mon[38032]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 71 MiB/s wr, 10.71k op/s 2024-03-22T19:17:23.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:23 smithi167 ceph-mon[47728]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 71 MiB/s wr, 10.71k op/s 2024-03-22T19:17:25.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:25 smithi086 ceph-mon[38032]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 80 MiB/s wr, 12.19k op/s 2024-03-22T19:17:25.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:25 smithi167 ceph-mon[47728]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 80 MiB/s wr, 12.19k op/s 2024-03-22T19:17:25.975 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:25 smithi131 ceph-mon[78559]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 80 MiB/s wr, 12.19k op/s 2024-03-22T19:17:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:27 smithi131 ceph-mon[78559]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 75 MiB/s wr, 11.61k op/s 2024-03-22T19:17:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:27 smithi086 ceph-mon[38032]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 75 MiB/s wr, 11.61k op/s 2024-03-22T19:17:27.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:27 smithi167 ceph-mon[47728]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 75 MiB/s wr, 11.61k op/s 2024-03-22T19:17:29.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:29 smithi131 ceph-mon[78559]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 75 MiB/s wr, 11.76k op/s 2024-03-22T19:17:29.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:29 smithi086 ceph-mon[38032]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 75 MiB/s wr, 11.76k op/s 2024-03-22T19:17:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:29 smithi167 ceph-mon[47728]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 75 MiB/s wr, 11.76k op/s 2024-03-22T19:17:31.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:31 smithi131 ceph-mon[78559]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 75 MiB/s wr, 12.00k op/s 2024-03-22T19:17:31.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:31 smithi086 ceph-mon[38032]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 75 MiB/s wr, 12.00k op/s 2024-03-22T19:17:31.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:31 smithi167 ceph-mon[47728]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 75 MiB/s wr, 12.00k op/s 2024-03-22T19:17:33.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:33 smithi131 ceph-mon[78559]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 76 MiB/s wr, 12.18k op/s 2024-03-22T19:17:33.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:33 smithi086 ceph-mon[38032]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 76 MiB/s wr, 12.18k op/s 2024-03-22T19:17:33.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:33 smithi167 ceph-mon[47728]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 76 MiB/s wr, 12.18k op/s 2024-03-22T19:17:35.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:35 smithi131 ceph-mon[78559]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 72 MiB/s wr, 11.57k op/s 2024-03-22T19:17:35.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:35 smithi086 ceph-mon[38032]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 72 MiB/s wr, 11.57k op/s 2024-03-22T19:17:35.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:35 smithi167 ceph-mon[47728]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 72 MiB/s wr, 11.57k op/s 2024-03-22T19:17:37.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:37 smithi131 ceph-mon[78559]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 MiB/s wr, 11.20k op/s 2024-03-22T19:17:37.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:37 smithi086 ceph-mon[38032]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 MiB/s wr, 11.20k op/s 2024-03-22T19:17:37.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:37 smithi167 ceph-mon[47728]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 70 MiB/s wr, 11.20k op/s 2024-03-22T19:17:39.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:39 smithi131 ceph-mon[78559]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 71 MiB/s wr, 11.24k op/s 2024-03-22T19:17:39.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:39 smithi086 ceph-mon[38032]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 71 MiB/s wr, 11.24k op/s 2024-03-22T19:17:39.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:39 smithi167 ceph-mon[47728]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 71 MiB/s wr, 11.24k op/s 2024-03-22T19:17:41.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:41 smithi086 ceph-mon[38032]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 81 MiB/s wr, 12.57k op/s 2024-03-22T19:17:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:41 smithi167 ceph-mon[47728]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 81 MiB/s wr, 12.57k op/s 2024-03-22T19:17:41.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:41 smithi131 ceph-mon[78559]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 81 MiB/s wr, 12.57k op/s 2024-03-22T19:17:43.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:17:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:43 smithi086 ceph-mon[38032]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 80 MiB/s wr, 12.47k op/s 2024-03-22T19:17:43.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:43 smithi167 ceph-mon[47728]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 80 MiB/s wr, 12.47k op/s 2024-03-22T19:17:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 -- 172.21.15.86:0/3932297820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 msgr2=0x7f1f0010b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 --2- 172.21.15.86:0/3932297820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 0x7f1f0010b3b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009990 tx=0x7f1ef8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 -- 172.21.15.86:0/3932297820 shutdown_connections 2024-03-22T19:17:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 --2- 172.21.15.86:0/3932297820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 0x7f1f0010b3b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 --2- 172.21.15.86:0/3932297820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f00102f00 0x7f1f00103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 --2- 172.21.15.86:0/3932297820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f001040d0 0x7f1f00104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.836+0000 7f1f07800700 1 -- 172.21.15.86:0/3932297820 >> 172.21.15.86:0/3932297820 conn(0x7f1f000fe4c0 msgr2=0x7f1f00100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.837+0000 7f1f07800700 1 -- 172.21.15.86:0/3932297820 shutdown_connections 2024-03-22T19:17:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.837+0000 7f1f07800700 1 -- 172.21.15.86:0/3932297820 wait complete. 2024-03-22T19:17:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.838+0000 7f1f07800700 1 Processor -- start 2024-03-22T19:17:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.839+0000 7f1f07800700 1 -- start start 2024-03-22T19:17:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.839+0000 7f1f07800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 0x7f1f00194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.839+0000 7f1f07800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f001040d0 0x7f1f00194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.839+0000 7f1f07800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 0x7f1f0019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f04e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 0x7f1f00194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f07800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019c540 con 0x7f1f001040d0 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f04e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 0x7f1f00194a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:38202/0 (socket says 172.21.15.86:38202) 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f04e00700 1 -- 172.21.15.86:0/4109955013 learned_addr learned my addr 172.21.15.86:0/4109955013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f05800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 0x7f1f0019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001ad7e0 con 0x7f1f00102f00 2024-03-22T19:17:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001ad920 con 0x7f1f00104a90 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.840+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f001040d0 0x7f1f00194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.841+0000 7f1f04e00700 1 -- 172.21.15.86:0/4109955013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 msgr2=0x7f1f0019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.841+0000 7f1f04e00700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 0x7f1f0019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.841+0000 7f1f04e00700 1 -- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f001040d0 msgr2=0x7f1f00194f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.841+0000 7f1f04e00700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f001040d0 0x7f1f00194f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.841+0000 7f1f04e00700 1 -- 172.21.15.86:0/4109955013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80095f0 con 0x7f1f00102f00 2024-03-22T19:17:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.841+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f001040d0 0x7f1f00194f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:17:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.842+0000 7f1f04e00700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 0x7f1f00194a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0007f10 tx=0x7f1ef000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.842+0000 7f1efe000700 1 -- 172.21.15.86:0/4109955013 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0017070 con 0x7f1f00102f00 2024-03-22T19:17:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.843+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f001adac0 con 0x7f1f00102f00 2024-03-22T19:17:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.843+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f001ae020 con 0x7f1f00102f00 2024-03-22T19:17:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.843+0000 7f1efe000700 1 -- 172.21.15.86:0/4109955013 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef000f040 con 0x7f1f00102f00 2024-03-22T19:17:43.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.843+0000 7f1efe000700 1 -- 172.21.15.86:0/4109955013 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0017070 con 0x7f1f00102f00 2024-03-22T19:17:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.848+0000 7f1efe000700 1 -- 172.21.15.86:0/4109955013 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f1ef0003a00 con 0x7f1f00102f00 2024-03-22T19:17:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.854+0000 7f1f1b3beb80 1 Processor -- start 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.854+0000 7f1efe000700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1ee40a0bf0 0x7f1ee40a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.854+0000 7f1f1b3beb80 1 -- start start 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.854+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 0x563e5e664040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.854+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e664e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.855+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.855+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e5bc990 con 0x563e5e664580 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.855+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e5ba350 con 0x563e5e663c20 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.855+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e5b5c40 con 0x563e5e665420 2024-03-22T19:17:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.855+0000 7f1efe000700 1 -- 172.21.15.86:0/4109955013 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1ef00cb3e0 con 0x7f1f00102f00 2024-03-22T19:17:43.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 0x563e5e664040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 0x563e5e664040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:38210/0 (socket says 172.21.15.86:38210) 2024-03-22T19:17:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 -- 172.21.15.86:0/1615503626 learned_addr learned my addr 172.21.15.86:0/1615503626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f05800700 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 -- 172.21.15.86:0/1615503626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 msgr2=0x563e5e6658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 -- 172.21.15.86:0/1615503626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 msgr2=0x563e5e664e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e664e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 -- 172.21.15.86:0/1615503626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5e4373b0 con 0x563e5e663c20 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.856+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 0x563e5e664040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00989f0 tx=0x7f1ef0098830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582907f37aef47ea server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.857+0000 7f1ef6000700 1 -- 172.21.15.86:0/1615503626 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0099160 con 0x563e5e663c20 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.857+0000 7f1ef6000700 1 -- 172.21.15.86:0/1615503626 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef0098a60 con 0x563e5e663c20 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1615503626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 msgr2=0x563e5e664040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 0x563e5e664040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00989f0 tx=0x7f1ef0098830 comp rx=0 tx=0).stop 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f05800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1ee40a0bf0 0x7f1ee40a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1615503626 shutdown_connections 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e663c20 0x563e5e664040 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00989f0 tx=0x7f1ef0098830 comp rx=0 tx=0).stop 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1615503626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e664e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.858+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1615503626 >> 172.21.15.86:0/1615503626 conn(0x563e5e5d9220 msgr2=0x563e5e65ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.860+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1615503626 shutdown_connections 2024-03-22T19:17:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.860+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1615503626 wait complete. 2024-03-22T19:17:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.861+0000 7f1f1b3beb80 1 -- start start 2024-03-22T19:17:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.861+0000 7f1f05800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1ee40a0bf0 0x7f1ee40a30b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x563e5e664c00 tx=0x7f1ef801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.861+0000 7f1f1b3beb80 1 Processor -- start 2024-03-22T19:17:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.862+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e66df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.862+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e665420 0x563e5e66e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.862+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e675870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e551d80 con 0x563e5e665420 2024-03-22T19:17:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e551a20 con 0x563e5e664580 2024-03-22T19:17:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e4396a0 con 0x563e5e673480 2024-03-22T19:17:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f1b3beb80 10 client.0 _ll_register_callbacks cb 0x563e5e3bd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:17:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f05800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e675870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e665420 0x563e5e66e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f05800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e675870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42786/0 (socket says 172.21.15.86:42786) 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.865+0000 7f1f05800700 1 -- 172.21.15.86:0/2689723156 learned_addr learned my addr 172.21.15.86:0/2689723156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.866+0000 7f1ef7e00700 1 -- 172.21.15.86:0/2689723156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 msgr2=0x563e5e675870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e675870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1ef7e00700 1 -- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 msgr2=0x563e5e66df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e66df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1ef7e00700 1 -- 172.21.15.86:0/2689723156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f001ae020 con 0x563e5e665420 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1f05800700 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e675870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e665420 0x563e5e66e450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009fd0 tx=0x7f1ee800e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235ff76fd55c3c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.868+0000 7f1eeea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:17:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.869+0000 7f1eeea00700 1 -- 172.21.15.86:0/2689723156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8011880 con 0x563e5e665420 2024-03-22T19:17:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.869+0000 7f1eeea00700 1 -- 172.21.15.86:0/2689723156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1ee800ee80 con 0x563e5e665420 2024-03-22T19:17:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.869+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e5e5ae700 con 0x563e5e665420 2024-03-22T19:17:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.869+0000 7f1eeea00700 1 -- 172.21.15.86:0/2689723156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee80119e0 con 0x563e5e665420 2024-03-22T19:17:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.869+0000 7f1f1b3beb80 10 client.16143 fetch_fsmap learned FSMap version 42 2024-03-22T19:17:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.869+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x563e5e3e3b60 con 0x563e5e665420 2024-03-22T19:17:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1eeea00700 1 -- 172.21.15.86:0/2689723156 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e5e5ae700 con 0x563e5e665420 2024-03-22T19:17:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1eeea00700 1 -- 172.21.15.86:0/2689723156 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ee800c070 con 0x563e5e665420 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1f1b3beb80 10 client.16143 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1f1b3beb80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1f1b3beb80 20 client.16143 populate_metadata read hostname 'smithi086' 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e67a1b0 0x563e5e67c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1f1b3beb80 4 client.16143 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.870+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e5e404cf0 con 0x563e5e67a1b0 2024-03-22T19:17:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.871+0000 7f1f04e00700 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e67a1b0 0x563e5e67c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.872+0000 7f1f04e00700 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e67a1b0 0x563e5e67c5d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3c78c559495177 server_cookie=2c7eda524663c2a4 in_seq=0 out_seq=0 2024-03-22T19:17:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.872+0000 7f1eeea00700 10 client.16143 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.873+0000 7f1eeea00700 1 -- 172.21.15.86:0/2689723156 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563e5e404cf0 con 0x563e5e67a1b0 2024-03-22T19:17:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.873+0000 7f1eeea00700 10 client.16143 handle_command_reply: tid=0 2024-03-22T19:17:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.873+0000 7f1f1b3beb80 1 client.16143 shutdown 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e665420 msgr2=0x563e5e66e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e665420 0x563e5e66e450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009fd0 tx=0x7f1ee800e750 comp rx=0 tx=0).stop 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 shutdown_connections 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e67a1b0 0x563e5e67c5d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e675870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e66df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/2689723156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e665420 0x563e5e66e450 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 >> 172.21.15.86:0/2689723156 conn(0x563e5e5d9220 msgr2=0x563e5e65e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1eeea00700 0 client.16143 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 shutdown_connections 2024-03-22T19:17:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.875+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/2689723156 wait complete. 2024-03-22T19:17:43.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.876+0000 7f1f1b3beb80 20 client.16143 trim_cache size 0 max 16384 2024-03-22T19:17:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.907+0000 7f1f1b3beb80 1 Processor -- start 2024-03-22T19:17:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.907+0000 7f1f1b3beb80 1 -- start start 2024-03-22T19:17:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e689ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e665420 0x563e5e665800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e668520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e66ee10 con 0x563e5e664580 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e65b5e0 con 0x563e5e665420 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e6776a0 con 0x563e5e673480 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e665420 0x563e5e665800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e665420 0x563e5e665800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:38230/0 (socket says 172.21.15.86:38230) 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.908+0000 7f1ef7e00700 1 -- 172.21.15.86:0/1080500312 learned_addr learned my addr 172.21.15.86:0/1080500312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e689ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f05800700 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e668520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f04e00700 1 -- 172.21.15.86:0/1080500312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 msgr2=0x563e5e668520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e668520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f04e00700 1 -- 172.21.15.86:0/1080500312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e665420 msgr2=0x563e5e665800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e665420 0x563e5e665800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.909+0000 7f1f04e00700 1 -- 172.21.15.86:0/1080500312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5e57bea0 con 0x563e5e664580 2024-03-22T19:17:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.911+0000 7f1f04e00700 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e689ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x563e5e66c240 tx=0x7f1ef0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37ec2f0e068c409 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.912+0000 7f1ef6a00700 1 -- 172.21.15.86:0/1080500312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0099160 con 0x563e5e664580 2024-03-22T19:17:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.912+0000 7f1ef6a00700 1 -- 172.21.15.86:0/1080500312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef009c890 con 0x563e5e664580 2024-03-22T19:17:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.912+0000 7f1ef6a00700 1 -- 172.21.15.86:0/1080500312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00224b0 con 0x563e5e664580 2024-03-22T19:17:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.913+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1080500312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 msgr2=0x563e5e689ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.913+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e689ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x563e5e66c240 tx=0x7f1ef0000f40 comp rx=0 tx=0).stop 2024-03-22T19:17:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.913+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1080500312 shutdown_connections 2024-03-22T19:17:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.913+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e673480 0x563e5e668520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.914+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e664580 0x563e5e689ad0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.914+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/1080500312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e665420 0x563e5e665800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.914+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1080500312 >> 172.21.15.86:0/1080500312 conn(0x563e5e5d9220 msgr2=0x563e5e65e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.914+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1080500312 shutdown_connections 2024-03-22T19:17:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.914+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/1080500312 wait complete. 2024-03-22T19:17:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.915+0000 7f1f1b3beb80 1 -- start start 2024-03-22T19:17:43.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.915+0000 7f1f1b3beb80 1 Processor -- start 2024-03-22T19:17:43.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e6c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e673480 0x563e5e6cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e689e60 con 0x563e5e673480 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e6cb250 con 0x563e5e664580 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5e6cb390 con 0x563e5e665420 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f1b3beb80 10 client.0 _ll_register_callbacks cb 0x7f1ee8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f04e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e6c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f04e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e6c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:38242/0 (socket says 172.21.15.86:38242) 2024-03-22T19:17:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1f04e00700 1 -- 172.21.15.86:0/196186972 learned_addr learned my addr 172.21.15.86:0/196186972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:17:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.916+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1f04e00700 1 -- 172.21.15.86:0/196186972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 msgr2=0x563e5e6c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1f04e00700 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1f04e00700 1 -- 172.21.15.86:0/196186972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e673480 msgr2=0x563e5e6cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:17:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1f04e00700 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e673480 0x563e5e6cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1f04e00700 1 -- 172.21.15.86:0/196186972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e5e4373b0 con 0x563e5e664580 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1f04e00700 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e6c6270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00989f0 tx=0x7f1ef0098af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439c918629baeeaf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.917+0000 7f1eeea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.918+0000 7f1eeea00700 1 -- 172.21.15.86:0/196186972 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00a0ee0 con 0x563e5e664580 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.918+0000 7f1eeea00700 1 -- 172.21.15.86:0/196186972 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f1ef00820b0 con 0x563e5e664580 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.918+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e5e656ad0 con 0x563e5e664580 2024-03-22T19:17:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.918+0000 7f1eeea00700 1 -- 172.21.15.86:0/196186972 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ef00a2020 con 0x563e5e664580 2024-03-22T19:17:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.918+0000 7f1f1b3beb80 10 client.26285 fetch_fsmap learned FSMap version 42 2024-03-22T19:17:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.918+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x563e5e43bcd0 con 0x563e5e664580 2024-03-22T19:17:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.919+0000 7f1eeea00700 1 -- 172.21.15.86:0/196186972 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef0082270 con 0x563e5e664580 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1eeea00700 1 -- 172.21.15.86:0/196186972 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ef007e070 con 0x563e5e664580 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1f1b3beb80 10 client.26285 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1f1b3beb80 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1f1b3beb80 20 client.26285 populate_metadata read hostname 'smithi086' 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e6cc870 0x563e5e6cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1f1b3beb80 4 client.26285 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:17:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.920+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e5e656ad0 con 0x563e5e6cc870 2024-03-22T19:17:43.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.921+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e6cc870 0x563e5e6cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:17:43.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.921+0000 7f1ef7e00700 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e6cc870 0x563e5e6cec60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be10fff523e76d9e server_cookie=452ad189679afa25 in_seq=0 out_seq=0 2024-03-22T19:17:43.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.922+0000 7f1eeea00700 10 client.26285 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:43.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.922+0000 7f1eeea00700 1 -- 172.21.15.86:0/196186972 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563e5e656ad0 con 0x563e5e6cc870 2024-03-22T19:17:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.922+0000 7f1eeea00700 10 client.26285 handle_command_reply: tid=0 2024-03-22T19:17:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.922+0000 7f1f1b3beb80 1 client.26285 shutdown 2024-03-22T19:17:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 msgr2=0x563e5e6c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e6c6270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00989f0 tx=0x7f1ef0098af0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 shutdown_connections 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x563e5e6cc870 0x563e5e6cec60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e5e673480 0x563e5e6cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e5e664580 0x563e5e6c6270 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1eeea00700 0 client.26285 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 --2- 172.21.15.86:0/196186972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e5e665420 0x563e5e6c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.923+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 >> 172.21.15.86:0/196186972 conn(0x563e5e5d9220 msgr2=0x563e5e668520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.924+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 shutdown_connections 2024-03-22T19:17:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.924+0000 7f1f1b3beb80 1 -- 172.21.15.86:0/196186972 wait complete. 2024-03-22T19:17:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.924+0000 7f1f1b3beb80 20 client.26285 trim_cache size 0 max 16384 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "c744d376-abbf-46e1-aef8-f02b44a2f81f": { 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:17:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.925+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1ee40a0bf0 msgr2=0x7f1ee40a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.925+0000 7f1f07800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1ee40a0bf0 0x7f1ee40a30b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x563e5e664c00 tx=0x7f1ef801a040 comp rx=0 tx=0).stop 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.925+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 msgr2=0x7f1f00194a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.925+0000 7f1f07800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 0x7f1f00194a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0007f10 tx=0x7f1ef000d3b0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 shutdown_connections 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f1ee40a0bf0 0x7f1ee40a30b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f00104a90 0x7f1f0019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f00102f00 0x7f1f00194a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 --2- 172.21.15.86:0/4109955013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f001040d0 0x7f1f00194f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:17:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 >> 172.21.15.86:0/4109955013 conn(0x7f1f000fe4c0 msgr2=0x7f1f000772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:17:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.926+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 shutdown_connections 2024-03-22T19:17:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:17:43.927+0000 7f1f07800700 1 -- 172.21.15.86:0/4109955013 wait complete. 2024-03-22T19:17:43.967 INFO:tasks.cephfs.filesystem:scrub status for tag:c744d376-abbf-46e1-aef8-f02b44a2f81f - {'path': '/', 'tag': 'c744d376-abbf-46e1-aef8-f02b44a2f81f', 'options': 'recursive,force'} 2024-03-22T19:17:43.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:43 smithi131 ceph-mon[78559]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 80 MiB/s wr, 12.47k op/s 2024-03-22T19:17:45.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:45 smithi086 ceph-mon[38032]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 70 MiB/s wr, 10.87k op/s 2024-03-22T19:17:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:45 smithi167 ceph-mon[47728]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 70 MiB/s wr, 10.87k op/s 2024-03-22T19:17:45.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:45 smithi131 ceph-mon[78559]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 70 MiB/s wr, 10.87k op/s 2024-03-22T19:17:47.727 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:47 smithi131 ceph-mon[78559]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 MiB/s wr, 13.17k op/s 2024-03-22T19:17:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:47 smithi086 ceph-mon[38032]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 MiB/s wr, 13.17k op/s 2024-03-22T19:17:47.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:47 smithi167 ceph-mon[47728]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 MiB/s wr, 13.17k op/s 2024-03-22T19:17:49.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:49 smithi131 ceph-mon[78559]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 74 MiB/s wr, 11.44k op/s 2024-03-22T19:17:49.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:49 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:17:49.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:49 smithi086 ceph-mon[38032]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 74 MiB/s wr, 11.44k op/s 2024-03-22T19:17:49.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:49 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:17:49.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:49 smithi167 ceph-mon[47728]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 74 MiB/s wr, 11.44k op/s 2024-03-22T19:17:49.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:49 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:17:51.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:51 smithi131 ceph-mon[78559]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 79 MiB/s wr, 12.27k op/s 2024-03-22T19:17:51.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:17:51.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:17:51.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:51 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:17:51.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:51 smithi086 ceph-mon[38032]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 79 MiB/s wr, 12.27k op/s 2024-03-22T19:17:51.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:17:51.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:17:51.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:51 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:17:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:51 smithi167 ceph-mon[47728]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 79 MiB/s wr, 12.27k op/s 2024-03-22T19:17:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:17:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:17:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:51 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:17:53.726 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:53 smithi131 ceph-mon[78559]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 73 MiB/s wr, 11.30k op/s 2024-03-22T19:17:53.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:53 smithi086 ceph-mon[38032]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 73 MiB/s wr, 11.30k op/s 2024-03-22T19:17:53.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:53 smithi167 ceph-mon[47728]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 73 MiB/s wr, 11.30k op/s 2024-03-22T19:17:55.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:55 smithi131 ceph-mon[78559]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 63 MiB/s wr, 9.84k op/s 2024-03-22T19:17:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:55 smithi086 ceph-mon[38032]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 63 MiB/s wr, 9.84k op/s 2024-03-22T19:17:55.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:55 smithi167 ceph-mon[47728]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 63 MiB/s wr, 9.84k op/s 2024-03-22T19:17:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:57 smithi131 ceph-mon[78559]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 MiB/s wr, 12.38k op/s 2024-03-22T19:17:57.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:57 smithi086 ceph-mon[38032]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 MiB/s wr, 12.38k op/s 2024-03-22T19:17:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:57 smithi167 ceph-mon[47728]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 MiB/s wr, 12.38k op/s 2024-03-22T19:17:59.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:17:59 smithi131 ceph-mon[78559]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 MiB/s wr, 10.83k op/s 2024-03-22T19:17:59.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:17:59 smithi086 ceph-mon[38032]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 MiB/s wr, 10.83k op/s 2024-03-22T19:17:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:17:59 smithi167 ceph-mon[47728]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 MiB/s wr, 10.83k op/s 2024-03-22T19:18:01.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:01 smithi131 ceph-mon[78559]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 MiB/s wr, 12.38k op/s 2024-03-22T19:18:01.763 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:01 smithi086 ceph-mon[38032]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 MiB/s wr, 12.38k op/s 2024-03-22T19:18:01.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:01 smithi167 ceph-mon[47728]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 MiB/s wr, 12.38k op/s 2024-03-22T19:18:03.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:03 smithi131 ceph-mon[78559]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 71 MiB/s wr, 11.58k op/s 2024-03-22T19:18:03.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:03 smithi086 ceph-mon[38032]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 71 MiB/s wr, 11.58k op/s 2024-03-22T19:18:03.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:03 smithi167 ceph-mon[47728]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 71 MiB/s wr, 11.58k op/s 2024-03-22T19:18:05.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:05 smithi131 ceph-mon[78559]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 68 MiB/s wr, 11.15k op/s 2024-03-22T19:18:05.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:05 smithi086 ceph-mon[38032]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 68 MiB/s wr, 11.15k op/s 2024-03-22T19:18:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:05 smithi167 ceph-mon[47728]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 68 MiB/s wr, 11.15k op/s 2024-03-22T19:18:07.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:07 smithi131 ceph-mon[78559]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 76 MiB/s wr, 12.45k op/s 2024-03-22T19:18:07.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:07 smithi086 ceph-mon[38032]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 76 MiB/s wr, 12.45k op/s 2024-03-22T19:18:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:07 smithi167 ceph-mon[47728]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 76 MiB/s wr, 12.45k op/s 2024-03-22T19:18:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:09 smithi131 ceph-mon[78559]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 MiB/s wr, 11.66k op/s 2024-03-22T19:18:09.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:09 smithi086 ceph-mon[38032]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 MiB/s wr, 11.66k op/s 2024-03-22T19:18:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:09 smithi167 ceph-mon[47728]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 MiB/s wr, 11.66k op/s 2024-03-22T19:18:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:11 smithi131 ceph-mon[78559]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 67 MiB/s wr, 10.76k op/s 2024-03-22T19:18:11.763 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:11 smithi086 ceph-mon[38032]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 67 MiB/s wr, 10.76k op/s 2024-03-22T19:18:11.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:11 smithi167 ceph-mon[47728]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 67 MiB/s wr, 10.76k op/s 2024-03-22T19:18:13.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:13 smithi131 ceph-mon[78559]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 71 MiB/s wr, 11.45k op/s 2024-03-22T19:18:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:13 smithi086 ceph-mon[38032]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 71 MiB/s wr, 11.45k op/s 2024-03-22T19:18:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:13 smithi167 ceph-mon[47728]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 71 MiB/s wr, 11.45k op/s 2024-03-22T19:18:13.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:18:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.408+0000 7ff595e00700 1 -- 172.21.15.86:0/1232851665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900fea20 msgr2=0x7ff5900fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.408+0000 7ff595e00700 1 --2- 172.21.15.86:0/1232851665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900fea20 0x7ff5900fee40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff58800b0a0 tx=0x7ff58800b3b0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 -- 172.21.15.86:0/1232851665 shutdown_connections 2024-03-22T19:18:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 --2- 172.21.15.86:0/1232851665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590100510 0x7ff590104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 --2- 172.21.15.86:0/1232851665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900fea20 0x7ff5900fee40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 --2- 172.21.15.86:0/1232851665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5900ffb50 0x7ff5900fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 -- 172.21.15.86:0/1232851665 >> 172.21.15.86:0/1232851665 conn(0x7ff5900fa340 msgr2=0x7ff5900fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 -- 172.21.15.86:0/1232851665 shutdown_connections 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.409+0000 7ff595e00700 1 -- 172.21.15.86:0/1232851665 wait complete. 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.412+0000 7ff595e00700 1 Processor -- start 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.412+0000 7ff595e00700 1 -- start start 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.413+0000 7ff595e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 0x7ff59019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff595e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5900ffb50 0x7ff59019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff595e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff590100510 0x7ff5901a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff594a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 0x7ff59019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff594a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 0x7ff59019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39834/0 (socket says 172.21.15.86:39834) 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff595e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901a4e30 con 0x7ff590100510 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff595e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901a4f70 con 0x7ff5900fea20 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff595e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901a5100 con 0x7ff5900ffb50 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.414+0000 7ff594a00700 1 -- 172.21.15.86:0/4284133132 learned_addr learned my addr 172.21.15.86:0/4284133132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:14.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.415+0000 7ff58fe00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5900ffb50 0x7ff59019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.415+0000 7ff595400700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff590100510 0x7ff5901a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.416+0000 7ff594a00700 1 -- 172.21.15.86:0/4284133132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5900ffb50 msgr2=0x7ff59019d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.416+0000 7ff594a00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5900ffb50 0x7ff59019d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.416+0000 7ff594a00700 1 -- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff590100510 msgr2=0x7ff5901a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.416+0000 7ff594a00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff590100510 0x7ff5901a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.416+0000 7ff594a00700 1 -- 172.21.15.86:0/4284133132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff588009d00 con 0x7ff5900fea20 2024-03-22T19:18:14.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.421+0000 7ff594a00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 0x7ff59019d350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff588009850 tx=0x7ff588004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:14.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.422+0000 7ff58d600700 1 -- 172.21.15.86:0/4284133132 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800e040 con 0x7ff5900fea20 2024-03-22T19:18:14.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.422+0000 7ff58d600700 1 -- 172.21.15.86:0/4284133132 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff588009440 con 0x7ff5900fea20 2024-03-22T19:18:14.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.422+0000 7ff58d600700 1 -- 172.21.15.86:0/4284133132 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588004020 con 0x7ff5900fea20 2024-03-22T19:18:14.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.422+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5901a5380 con 0x7ff5900fea20 2024-03-22T19:18:14.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.423+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5901a57b0 con 0x7ff5900fea20 2024-03-22T19:18:14.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.436+0000 7ff58d600700 1 -- 172.21.15.86:0/4284133132 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff58801d690 con 0x7ff5900fea20 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.438+0000 7ff5aafa9b80 1 Processor -- start 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.439+0000 7ff58d600700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5740a0ae0 0x7ff5740a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.439+0000 7ff58d600700 1 -- 172.21.15.86:0/4284133132 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff5880cd4b0 con 0x7ff5900fea20 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.440+0000 7ff595400700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5740a0ae0 0x7ff5740a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.440+0000 7ff5aafa9b80 1 -- start start 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.441+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.441+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d0d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.441+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.441+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8c64990 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.441+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8c62350 con 0x55b0f8d0d020 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.441+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8c5dc40 con 0x55b0f8d0bde0 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.442+0000 7ff58fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.442+0000 7ff58fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d0d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39840/0 (socket says 172.21.15.86:39840) 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.442+0000 7ff58fe00700 1 -- 172.21.15.86:0/3237467217 learned_addr learned my addr 172.21.15.86:0/3237467217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff58fe00700 1 -- 172.21.15.86:0/3237467217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0bde0 msgr2=0x55b0f8d0c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff58fe00700 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff58fe00700 1 -- 172.21.15.86:0/3237467217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 msgr2=0x55b0f8d14270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff58fe00700 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff58fe00700 1 -- 172.21.15.86:0/3237467217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0f8adf3b0 con 0x55b0f8d0d020 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff595400700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5740a0ae0 0x7ff5740a2fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009990 tx=0x7ff57c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.444+0000 7ff58fe00700 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d0d4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff58000bd70 tx=0x7ff5800106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a1c6348b327a37 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.445+0000 7ff585c00700 1 -- 172.21.15.86:0/3237467217 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580010ec0 con 0x55b0f8d0d020 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.447+0000 7ff585c00700 1 -- 172.21.15.86:0/3237467217 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff580015070 con 0x55b0f8d0d020 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.447+0000 7ff585c00700 1 -- 172.21.15.86:0/3237467217 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800117c0 con 0x55b0f8d0d020 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/3237467217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 msgr2=0x55b0f8d0d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d0d4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff58000bd70 tx=0x7ff5800106b0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/3237467217 shutdown_connections 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/3237467217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d0d4a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.448+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/3237467217 >> 172.21.15.86:0/3237467217 conn(0x55b0f8c81220 msgr2=0x55b0f8d06e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.449+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/3237467217 shutdown_connections 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.449+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/3237467217 wait complete. 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.449+0000 7ff5aafa9b80 1 -- start start 2024-03-22T19:18:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.450+0000 7ff5aafa9b80 1 Processor -- start 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 0x55b0f8d161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d1d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8bf9d80 con 0x55b0f8d0bde0 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8bf9a20 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8ae16a0 con 0x55b0f8d0d020 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff5aafa9b80 10 client.0 _ll_register_callbacks cb 0x55b0f8a653a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff595400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d1d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff595400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d1d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39848/0 (socket says 172.21.15.86:39848) 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.451+0000 7ff595400700 1 -- 172.21.15.86:0/2712406544 learned_addr learned my addr 172.21.15.86:0/2712406544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.452+0000 7ff595400700 1 -- 172.21.15.86:0/2712406544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 msgr2=0x55b0f8d161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.452+0000 7ff595400700 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 0x55b0f8d161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.452+0000 7ff595400700 1 -- 172.21.15.86:0/2712406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0bde0 msgr2=0x55b0f8d15c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.452+0000 7ff595400700 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.452+0000 7ff595400700 1 -- 172.21.15.86:0/2712406544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58000b9a0 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.455+0000 7ff595400700 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d1d200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0154f0 tx=0x7ff57c004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cc7c5cd0c19d06 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.455+0000 7ff57a000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.459+0000 7ff57a000700 1 -- 172.21.15.86:0/2712406544 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c015a90 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.459+0000 7ff57a000700 1 -- 172.21.15.86:0/2712406544 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff57c015bf0 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.460+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0f8cfec30 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.465+0000 7ff57a000700 1 -- 172.21.15.86:0/2712406544 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff57c015db0 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.466+0000 7ff5aafa9b80 10 client.26294 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.466+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b0f8a8bb60 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.472+0000 7ff57a000700 1 -- 172.21.15.86:0/2712406544 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0f8cfec30 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.472+0000 7ff57a000700 1 -- 172.21.15.86:0/2712406544 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff57c01f5b0 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.473+0000 7ff5aafa9b80 10 client.26294 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:14.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.473+0000 7ff5aafa9b80 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:14.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.473+0000 7ff5aafa9b80 20 client.26294 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:14.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.473+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d21ae0 0x55b0f8d23f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.473+0000 7ff5aafa9b80 4 client.26294 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:14.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.473+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0f8c56700 con 0x55b0f8d21ae0 2024-03-22T19:18:14.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.474+0000 7ff594a00700 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d21ae0 0x55b0f8d23f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.486+0000 7ff594a00700 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d21ae0 0x55b0f8d23f00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e2e53d3ae7d40d server_cookie=a7fd6270b8a0bf86 in_seq=0 out_seq=0 2024-03-22T19:18:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.488+0000 7ff57a000700 10 client.26294 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.488+0000 7ff57a000700 1 -- 172.21.15.86:0/2712406544 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b0f8c56700 con 0x55b0f8d21ae0 2024-03-22T19:18:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.488+0000 7ff57a000700 10 client.26294 handle_command_reply: tid=0 2024-03-22T19:18:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.488+0000 7ff5aafa9b80 1 client.26294 shutdown 2024-03-22T19:18:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.488+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 msgr2=0x55b0f8d1d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.488+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d1d200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0154f0 tx=0x7ff57c004860 comp rx=0 tx=0).stop 2024-03-22T19:18:14.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 shutdown_connections 2024-03-22T19:18:14.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d21ae0 0x55b0f8d23f00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d1d200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d15c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2712406544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 0x55b0f8d161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 >> 172.21.15.86:0/2712406544 conn(0x55b0f8c81220 msgr2=0x55b0f8d06350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff57a000700 0 client.26294 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 shutdown_connections 2024-03-22T19:18:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.491+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2712406544 wait complete. 2024-03-22T19:18:14.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.495+0000 7ff5aafa9b80 20 client.26294 trim_cache size 0 max 16384 2024-03-22T19:18:14.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.549+0000 7ff5aafa9b80 1 Processor -- start 2024-03-22T19:18:14.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.551+0000 7ff5aafa9b80 1 -- start start 2024-03-22T19:18:14.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.551+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.551+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 0x55b0f8d13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.551+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff595400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8c56700 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8d085c0 con 0x55b0f8d0bde0 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8d03740 con 0x55b0f8d0d020 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff594a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff594a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d1b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39864/0 (socket says 172.21.15.86:39864) 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.552+0000 7ff594a00700 1 -- 172.21.15.86:0/2821362549 learned_addr learned my addr 172.21.15.86:0/2821362549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.553+0000 7ff594a00700 1 -- 172.21.15.86:0/2821362549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 msgr2=0x55b0f8d13ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.553+0000 7ff594a00700 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 0x55b0f8d13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.553+0000 7ff594a00700 1 -- 172.21.15.86:0/2821362549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 msgr2=0x55b0f8d14420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.555+0000 7ff594a00700 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.555+0000 7ff594a00700 1 -- 172.21.15.86:0/2821362549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0f8c23ea0 con 0x55b0f8d0bde0 2024-03-22T19:18:14.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.556+0000 7ff595400700 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:18:14.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.564+0000 7ff594a00700 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d1b1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff58809abf0 tx=0x7ff588004ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da33beb78d9077da server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:14.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.565+0000 7ff586600700 1 -- 172.21.15.86:0/2821362549 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58809a5c0 con 0x55b0f8d0bde0 2024-03-22T19:18:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.567+0000 7ff586600700 1 -- 172.21.15.86:0/2821362549 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58809b360 con 0x55b0f8d0bde0 2024-03-22T19:18:14.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.567+0000 7ff586600700 1 -- 172.21.15.86:0/2821362549 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588098e30 con 0x55b0f8d0bde0 2024-03-22T19:18:14.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.568+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2821362549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 msgr2=0x55b0f8d1b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.568+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d1b1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff58809abf0 tx=0x7ff588004ea0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.569+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2821362549 shutdown_connections 2024-03-22T19:18:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d14420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0bde0 0x55b0f8d1b1b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/2821362549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d020 0x55b0f8d13ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2821362549 >> 172.21.15.86:0/2821362549 conn(0x55b0f8c81220 msgr2=0x55b0f8d15190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2821362549 shutdown_connections 2024-03-22T19:18:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/2821362549 wait complete. 2024-03-22T19:18:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.570+0000 7ff5aafa9b80 1 -- start start 2024-03-22T19:18:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.571+0000 7ff5aafa9b80 1 Processor -- start 2024-03-22T19:18:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.572+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d6fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff5aafa9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 0x55b0f8d724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8d72bb0 con 0x55b0f8d70050 2024-03-22T19:18:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8d72d20 con 0x55b0f8d0d020 2024-03-22T19:18:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff5aafa9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f8d72ee0 con 0x55b0f8d0d9e0 2024-03-22T19:18:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff595400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 0x55b0f8d724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.573+0000 7ff5aafa9b80 10 client.0 _ll_register_callbacks cb 0x7ff57c00ea60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.574+0000 7ff595400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 0x55b0f8d724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57766/0 (socket says 172.21.15.86:57766) 2024-03-22T19:18:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.574+0000 7ff595400700 1 -- 172.21.15.86:0/714066858 learned_addr learned my addr 172.21.15.86:0/714066858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.575+0000 7ff595400700 1 -- 172.21.15.86:0/714066858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d9e0 msgr2=0x55b0f8d6fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.575+0000 7ff595400700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d6fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.575+0000 7ff594a00700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.577+0000 7ff595400700 1 -- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 msgr2=0x55b0f8d6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:14.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.577+0000 7ff595400700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.577+0000 7ff595400700 1 -- 172.21.15.86:0/714066858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5901a57b0 con 0x55b0f8d70050 2024-03-22T19:18:14.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.577+0000 7ff594a00700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:18:14.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.579+0000 7ff57a000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:14.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.579+0000 7ff595400700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 0x55b0f8d724a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0154f0 tx=0x7ff57c01f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df795193add3434 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.581+0000 7ff57a000700 1 -- 172.21.15.86:0/714066858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c015c20 con 0x55b0f8d70050 2024-03-22T19:18:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.581+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff580015070 con 0x55b0f8d70050 2024-03-22T19:18:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.581+0000 7ff57a000700 1 -- 172.21.15.86:0/714066858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff57c01fe40 con 0x55b0f8d70050 2024-03-22T19:18:14.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.582+0000 7ff57a000700 1 -- 172.21.15.86:0/714066858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff57c015c20 con 0x55b0f8d70050 2024-03-22T19:18:14.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.583+0000 7ff5aafa9b80 10 client.16158 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:14.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.583+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b0f8ae3cd0 con 0x55b0f8d70050 2024-03-22T19:18:14.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.584+0000 7ff57a000700 1 -- 172.21.15.86:0/714066858 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff580015070 con 0x55b0f8d70050 2024-03-22T19:18:14.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.584+0000 7ff57a000700 1 -- 172.21.15.86:0/714066858 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff57c0135f0 con 0x55b0f8d70050 2024-03-22T19:18:14.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.586+0000 7ff5aafa9b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:14.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.586+0000 7ff5aafa9b80 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:14.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.586+0000 7ff5aafa9b80 20 client.16158 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:14.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.587+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d74910 0x55b0f8d76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:14.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.587+0000 7ff594a00700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d74910 0x55b0f8d76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:14.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.587+0000 7ff5aafa9b80 4 client.16158 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:18:14.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.587+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0f8aaccf0 con 0x55b0f8d74910 2024-03-22T19:18:14.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.594+0000 7ff594a00700 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d74910 0x55b0f8d76d30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f45929ce45ae0b server_cookie=73d748ab266c417f in_seq=0 out_seq=0 2024-03-22T19:18:14.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.594+0000 7ff57a000700 10 client.16158 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:14.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.600+0000 7ff57a000700 1 -- 172.21.15.86:0/714066858 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b0f8aaccf0 con 0x55b0f8d74910 2024-03-22T19:18:14.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.600+0000 7ff57a000700 10 client.16158 handle_command_reply: tid=0 2024-03-22T19:18:14.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 client.16158 shutdown 2024-03-22T19:18:14.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 msgr2=0x55b0f8d724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 0x55b0f8d724a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0154f0 tx=0x7ff57c01f6b0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 shutdown_connections 2024-03-22T19:18:14.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55b0f8d74910 0x55b0f8d76d30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f8d70050 0x55b0f8d724a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0f8d0d020 0x55b0f8d6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 --2- 172.21.15.86:0/714066858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f8d0d9e0 0x55b0f8d6fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.601+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 >> 172.21.15.86:0/714066858 conn(0x55b0f8c81220 msgr2=0x55b0f8d13ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:14.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.602+0000 7ff57a000700 0 client.16158 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:14.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.602+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 shutdown_connections 2024-03-22T19:18:14.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.604+0000 7ff5aafa9b80 1 -- 172.21.15.86:0/714066858 wait complete. 2024-03-22T19:18:14.606 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:18:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-22T19:18:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:18:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "c744d376-abbf-46e1-aef8-f02b44a2f81f": { 2024-03-22T19:18:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:18:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c744d376-abbf-46e1-aef8-f02b44a2f81f", 2024-03-22T19:18:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:18:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:18:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:18:14.607 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:18:14.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.604+0000 7ff5aafa9b80 20 client.16158 trim_cache size 0 max 16384 2024-03-22T19:18:14.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5740a0ae0 msgr2=0x7ff5740a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5740a0ae0 0x7ff5740a2fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009990 tx=0x7ff57c009110 comp rx=0 tx=0).stop 2024-03-22T19:18:14.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 msgr2=0x7ff59019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:14.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 0x7ff59019d350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff588009850 tx=0x7ff588004c80 comp rx=0 tx=0).stop 2024-03-22T19:18:14.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 shutdown_connections 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff5740a0ae0 0x7ff5740a2fa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff590100510 0x7ff5901a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5900fea20 0x7ff59019d350 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 --2- 172.21.15.86:0/4284133132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5900ffb50 0x7ff59019d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.605+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 >> 172.21.15.86:0/4284133132 conn(0x7ff5900fa340 msgr2=0x7ff590100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.606+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 shutdown_connections 2024-03-22T19:18:14.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:14.607+0000 7ff595e00700 1 -- 172.21.15.86:0/4284133132 wait complete. 2024-03-22T19:18:14.661 INFO:tasks.cephfs.filesystem:scrub status for tag:c744d376-abbf-46e1-aef8-f02b44a2f81f - {'path': '/', 'tag': 'c744d376-abbf-46e1-aef8-f02b44a2f81f', 'options': 'recursive,force'} 2024-03-22T19:18:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:15 smithi167 ceph-mon[47728]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 71 MiB/s wr, 11.47k op/s 2024-03-22T19:18:15.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:15 smithi131 ceph-mon[78559]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 71 MiB/s wr, 11.47k op/s 2024-03-22T19:18:16.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:15 smithi086 ceph-mon[38032]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 71 MiB/s wr, 11.47k op/s 2024-03-22T19:18:17.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:17 smithi131 ceph-mon[78559]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 82 MiB/s wr, 13.22k op/s 2024-03-22T19:18:17.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:17 smithi086 ceph-mon[38032]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 82 MiB/s wr, 13.22k op/s 2024-03-22T19:18:17.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:17 smithi167 ceph-mon[47728]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 82 MiB/s wr, 13.22k op/s 2024-03-22T19:18:19.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:19 smithi131 ceph-mon[78559]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 72 MiB/s wr, 11.66k op/s 2024-03-22T19:18:19.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:19 smithi086 ceph-mon[38032]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 72 MiB/s wr, 11.66k op/s 2024-03-22T19:18:19.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:19 smithi167 ceph-mon[47728]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 72 MiB/s wr, 11.66k op/s 2024-03-22T19:18:21.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:21 smithi131 ceph-mon[78559]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 MiB/s wr, 11.85k op/s 2024-03-22T19:18:21.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:21 smithi086 ceph-mon[38032]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 MiB/s wr, 11.85k op/s 2024-03-22T19:18:21.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:21 smithi167 ceph-mon[47728]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 MiB/s wr, 11.85k op/s 2024-03-22T19:18:23.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:23 smithi131 ceph-mon[78559]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 81 MiB/s wr, 12.97k op/s 2024-03-22T19:18:23.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:23 smithi086 ceph-mon[38032]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 81 MiB/s wr, 12.97k op/s 2024-03-22T19:18:23.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:23 smithi167 ceph-mon[47728]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 81 MiB/s wr, 12.97k op/s 2024-03-22T19:18:25.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:25 smithi131 ceph-mon[78559]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 68 MiB/s wr, 10.84k op/s 2024-03-22T19:18:25.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:25 smithi086 ceph-mon[38032]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 68 MiB/s wr, 10.84k op/s 2024-03-22T19:18:25.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:25 smithi167 ceph-mon[47728]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 68 MiB/s wr, 10.84k op/s 2024-03-22T19:18:27.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:27 smithi131 ceph-mon[78559]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 MiB/s wr, 13.35k op/s 2024-03-22T19:18:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:27 smithi086 ceph-mon[38032]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 MiB/s wr, 13.35k op/s 2024-03-22T19:18:27.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:27 smithi167 ceph-mon[47728]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 MiB/s wr, 13.35k op/s 2024-03-22T19:18:29.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:29 smithi131 ceph-mon[78559]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 69 MiB/s wr, 10.75k op/s 2024-03-22T19:18:29.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:29 smithi086 ceph-mon[38032]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 69 MiB/s wr, 10.75k op/s 2024-03-22T19:18:29.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:29 smithi167 ceph-mon[47728]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 69 MiB/s wr, 10.75k op/s 2024-03-22T19:18:31.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:31 smithi131 ceph-mon[78559]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 73 MiB/s wr, 11.29k op/s 2024-03-22T19:18:31.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:31 smithi086 ceph-mon[38032]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 73 MiB/s wr, 11.29k op/s 2024-03-22T19:18:31.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:31 smithi167 ceph-mon[47728]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 73 MiB/s wr, 11.29k op/s 2024-03-22T19:18:33.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:33 smithi131 ceph-mon[78559]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 79 MiB/s wr, 12.21k op/s 2024-03-22T19:18:33.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:33 smithi086 ceph-mon[38032]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 79 MiB/s wr, 12.21k op/s 2024-03-22T19:18:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:33 smithi167 ceph-mon[47728]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 79 MiB/s wr, 12.21k op/s 2024-03-22T19:18:35.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:35 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:35.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:35 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:35.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:35 smithi086 ceph-mon[38032]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 71 MiB/s wr, 10.93k op/s 2024-03-22T19:18:35.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:35 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:35.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:35 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:35.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:35 smithi167 ceph-mon[47728]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 71 MiB/s wr, 10.93k op/s 2024-03-22T19:18:35.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:35 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:35.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:35 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:35.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:35 smithi131 ceph-mon[78559]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 71 MiB/s wr, 10.93k op/s 2024-03-22T19:18:37.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:37 smithi131 ceph-mon[78559]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 MiB/s wr, 10.17k op/s 2024-03-22T19:18:37.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:37 smithi086 ceph-mon[38032]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 MiB/s wr, 10.17k op/s 2024-03-22T19:18:37.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:37 smithi167 ceph-mon[47728]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 MiB/s wr, 10.17k op/s 2024-03-22T19:18:39.725 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:39 smithi131 ceph-mon[78559]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 60 MiB/s wr, 9.07k op/s 2024-03-22T19:18:39.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:39 smithi086 ceph-mon[38032]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 60 MiB/s wr, 9.07k op/s 2024-03-22T19:18:39.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:39 smithi167 ceph-mon[47728]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 60 MiB/s wr, 9.07k op/s 2024-03-22T19:18:41.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:41 smithi086 ceph-mon[38032]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 76 MiB/s wr, 11.74k op/s 2024-03-22T19:18:41.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:41 smithi167 ceph-mon[47728]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 76 MiB/s wr, 11.74k op/s 2024-03-22T19:18:41.976 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:41 smithi131 ceph-mon[78559]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 76 MiB/s wr, 11.74k op/s 2024-03-22T19:18:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:43 smithi086 ceph-mon[38032]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 MiB/s wr, 10.70k op/s 2024-03-22T19:18:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:43 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:18:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:43 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:18:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:43 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:18:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:43 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:18:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:43 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:18:43.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:43 smithi167 ceph-mon[47728]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 MiB/s wr, 10.70k op/s 2024-03-22T19:18:43.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:43 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:18:43.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:43 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:18:43.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:43 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:18:43.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:43 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:18:43.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:43 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:18:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:43 smithi131 ceph-mon[78559]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 MiB/s wr, 10.70k op/s 2024-03-22T19:18:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:43 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:18:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:43 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:18:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:43 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:18:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:43 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:18:43.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:43 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:18:44.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.035+0000 7fa4b4800700 1 -- 172.21.15.86:0/980885710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac0737b0 msgr2=0x7fa4ac073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.035+0000 7fa4b4800700 1 --2- 172.21.15.86:0/980885710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac0737b0 0x7fa4ac073c30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009990 tx=0x7fa49c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 -- 172.21.15.86:0/980885710 shutdown_connections 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 --2- 172.21.15.86:0/980885710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac108c10 0x7fa4ac10b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 --2- 172.21.15.86:0/980885710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac074d90 0x7fa4ac0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 --2- 172.21.15.86:0/980885710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac0737b0 0x7fa4ac073c30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 -- 172.21.15.86:0/980885710 >> 172.21.15.86:0/980885710 conn(0x7fa4ac0fc2f0 msgr2=0x7fa4ac0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 -- 172.21.15.86:0/980885710 shutdown_connections 2024-03-22T19:18:45.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.036+0000 7fa4b4800700 1 -- 172.21.15.86:0/980885710 wait complete. 2024-03-22T19:18:45.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.037+0000 7fa4b4800700 1 Processor -- start 2024-03-22T19:18:45.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.037+0000 7fa4b4800700 1 -- start start 2024-03-22T19:18:45.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b4800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0737b0 0x7fa4ac10fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b4800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 0x7fa4ac1103c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b4800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac108c10 0x7fa4ac1ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b4800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac110aa0 con 0x7fa4ac074d90 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b4800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1ad7f0 con 0x7fa4ac0737b0 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b4800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1ad960 con 0x7fa4ac108c10 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 0x7fa4ac1103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b2800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac108c10 0x7fa4ac1ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 0x7fa4ac1103c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59232/0 (socket says 172.21.15.86:59232) 2024-03-22T19:18:45.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.038+0000 7fa4b1400700 1 -- 172.21.15.86:0/2449432799 learned_addr learned my addr 172.21.15.86:0/2449432799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:45.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.039+0000 7fa4b1400700 1 -- 172.21.15.86:0/2449432799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac108c10 msgr2=0x7fa4ac1ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.039+0000 7fa4b1400700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac108c10 0x7fa4ac1ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.039+0000 7fa4b1400700 1 -- 172.21.15.86:0/2449432799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0737b0 msgr2=0x7fa4ac10fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.039+0000 7fa4b1400700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0737b0 0x7fa4ac10fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.039+0000 7fa4b1400700 1 -- 172.21.15.86:0/2449432799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c0095f0 con 0x7fa4ac074d90 2024-03-22T19:18:45.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.039+0000 7fa4b2800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac108c10 0x7fa4ac1ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:18:45.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.041+0000 7fa4b1400700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 0x7fa4ac1103c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0098e0 tx=0x7fa49c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.041+0000 7fa4a6a00700 1 -- 172.21.15.86:0/2449432799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c01d070 con 0x7fa4ac074d90 2024-03-22T19:18:45.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.041+0000 7fa4a6a00700 1 -- 172.21.15.86:0/2449432799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49c00fb80 con 0x7fa4ac074d90 2024-03-22T19:18:45.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.041+0000 7fa4a6a00700 1 -- 172.21.15.86:0/2449432799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c017650 con 0x7fa4ac074d90 2024-03-22T19:18:45.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.042+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac1adbe0 con 0x7fa4ac074d90 2024-03-22T19:18:45.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.042+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac1adf50 con 0x7fa4ac074d90 2024-03-22T19:18:45.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.047+0000 7fa4a6a00700 1 -- 172.21.15.86:0/2449432799 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa49c0177b0 con 0x7fa4ac074d90 2024-03-22T19:18:45.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.048+0000 7fa4c828bb80 1 Processor -- start 2024-03-22T19:18:45.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 -- start start 2024-03-22T19:18:45.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4a6a00700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4940a0b90 0x7fa4940a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693be1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be1580 0x55f693be1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be2420 0x55f693be28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4a6a00700 1 -- 172.21.15.86:0/2449432799 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa49c0cd670 con 0x7fa4ac074d90 2024-03-22T19:18:45.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693b39990 con 0x55f693be2420 2024-03-22T19:18:45.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693b37350 con 0x55f693be0c20 2024-03-22T19:18:45.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693b32c40 con 0x55f693be1580 2024-03-22T19:18:45.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be1580 0x55f693be1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.049+0000 7fa4b1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693be1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4940a0b90 0x7fa4940a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be1580 0x55f693be1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49066/0 (socket says 172.21.15.86:49066) 2024-03-22T19:18:45.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b1400700 1 -- 172.21.15.86:0/769075683 learned_addr learned my addr 172.21.15.86:0/769075683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:45.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be2420 0x55f693be28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 -- 172.21.15.86:0/769075683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be1580 msgr2=0x55f693be1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be1580 0x55f693be1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 -- 172.21.15.86:0/769075683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 msgr2=0x55f693be1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693be1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.050+0000 7fa4b2800700 1 -- 172.21.15.86:0/769075683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6939b43b0 con 0x55f693be2420 2024-03-22T19:18:45.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.051+0000 7fa4b2800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4940a0b90 0x7fa4940a3050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009fd0 tx=0x7fa4a8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.051+0000 7fa4b2800700 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be2420 0x55f693be28a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800ecb0 tx=0x7fa4a801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e503c45f209f10a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.051+0000 7fa49b000700 1 -- 172.21.15.86:0/769075683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8008570 con 0x55f693be2420 2024-03-22T19:18:45.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.052+0000 7fa49b000700 1 -- 172.21.15.86:0/769075683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4a800f040 con 0x55f693be2420 2024-03-22T19:18:45.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.052+0000 7fa49b000700 1 -- 172.21.15.86:0/769075683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8025ba0 con 0x55f693be2420 2024-03-22T19:18:45.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 -- 172.21.15.86:0/769075683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be2420 msgr2=0x55f693be28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be2420 0x55f693be28a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800ecb0 tx=0x7fa4a801d000 comp rx=0 tx=0).stop 2024-03-22T19:18:45.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 -- 172.21.15.86:0/769075683 shutdown_connections 2024-03-22T19:18:45.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be2420 0x55f693be28a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693be1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/769075683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be1580 0x55f693be1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 -- 172.21.15.86:0/769075683 >> 172.21.15.86:0/769075683 conn(0x55f693b56220 msgr2=0x55f693bdbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.053+0000 7fa4c828bb80 1 -- 172.21.15.86:0/769075683 shutdown_connections 2024-03-22T19:18:45.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.054+0000 7fa4c828bb80 1 -- 172.21.15.86:0/769075683 wait complete. 2024-03-22T19:18:45.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.054+0000 7fa4c828bb80 1 -- start start 2024-03-22T19:18:45.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 Processor -- start 2024-03-22T19:18:45.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be0c20 0x55f693beab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 0x55f693beb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693bf1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693aced80 con 0x55f693be0c20 2024-03-22T19:18:45.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693acea20 con 0x55f693be1580 2024-03-22T19:18:45.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.055+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6939b66a0 con 0x55f693be2420 2024-03-22T19:18:45.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4c828bb80 10 client.0 _ll_register_callbacks cb 0x55f69393a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:45.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 0x55f693beb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 0x55f693beb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33642/0 (socket says 172.21.15.86:33642) 2024-03-22T19:18:45.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 -- 172.21.15.86:0/907765082 learned_addr learned my addr 172.21.15.86:0/907765082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:45.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1e00700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be0c20 0x55f693beab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 -- 172.21.15.86:0/907765082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 msgr2=0x55f693bf1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693bf1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 -- 172.21.15.86:0/907765082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be0c20 msgr2=0x55f693beab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be0c20 0x55f693beab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1400700 1 -- 172.21.15.86:0/907765082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a800ea70 con 0x55f693be1580 2024-03-22T19:18:45.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.056+0000 7fa4b1e00700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be0c20 0x55f693beab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:18:45.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.057+0000 7fa4b1400700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 0x55f693beb090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00ffd0 tx=0x7fa49c005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b0ba8d1e8ecbee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.057+0000 7fa48f400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:45.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.057+0000 7fa48f400700 1 -- 172.21.15.86:0/907765082 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c09a470 con 0x55f693be1580 2024-03-22T19:18:45.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.057+0000 7fa48f400700 1 -- 172.21.15.86:0/907765082 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa49c09ad80 con 0x55f693be1580 2024-03-22T19:18:45.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.057+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f693bd3ad0 con 0x55f693be1580 2024-03-22T19:18:45.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.060+0000 7fa48f400700 1 -- 172.21.15.86:0/907765082 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa49c09ebb0 con 0x55f693be1580 2024-03-22T19:18:45.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.060+0000 7fa4c828bb80 10 client.26303 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:45.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.060+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f693960b60 con 0x55f693be1580 2024-03-22T19:18:45.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.061+0000 7fa48f400700 1 -- 172.21.15.86:0/907765082 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f693bd3ad0 con 0x55f693be1580 2024-03-22T19:18:45.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa48f400700 1 -- 172.21.15.86:0/907765082 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa49c0aebc0 con 0x55f693be1580 2024-03-22T19:18:45.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4c828bb80 10 client.26303 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:45.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4c828bb80 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:45.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4c828bb80 20 client.26303 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:45.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c074f0 0x55f693c09910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4c828bb80 4 client.26303 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:45.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f693b2b700 con 0x55f693c074f0 2024-03-22T19:18:45.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.062+0000 7fa4b1e00700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c074f0 0x55f693c09910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.065+0000 7fa48f400700 10 client.26303 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:45.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.065+0000 7fa4b1e00700 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c074f0 0x55f693c09910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe0c2f752bfcff3 server_cookie=a9cc2e7610ce8b78 in_seq=0 out_seq=0 2024-03-22T19:18:45.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.074+0000 7fa48f400700 1 -- 172.21.15.86:0/907765082 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f693b2b700 con 0x55f693c074f0 2024-03-22T19:18:45.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.074+0000 7fa48f400700 10 client.26303 handle_command_reply: tid=0 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 client.26303 shutdown 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 msgr2=0x55f693beb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 0x55f693beb090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00ffd0 tx=0x7fa49c005ad0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 shutdown_connections 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c074f0 0x55f693c09910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693bf1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be0c20 0x55f693beab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/907765082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be1580 0x55f693beb090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 >> 172.21.15.86:0/907765082 conn(0x55f693b56220 msgr2=0x55f693bdb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa48f400700 0 client.26303 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 shutdown_connections 2024-03-22T19:18:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 1 -- 172.21.15.86:0/907765082 wait complete. 2024-03-22T19:18:45.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.075+0000 7fa4c828bb80 20 client.26303 trim_cache size 0 max 16384 2024-03-22T19:18:45.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.102+0000 7fa4c828bb80 1 Processor -- start 2024-03-22T19:18:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.102+0000 7fa4c828bb80 1 -- start start 2024-03-22T19:18:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693c036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693be53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693bd6210 con 0x55f693be1580 2024-03-22T19:18:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693bea6c0 con 0x55f693be0c20 2024-03-22T19:18:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693bd8610 con 0x55f693be2420 2024-03-22T19:18:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4b1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693c036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4b1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693c036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33656/0 (socket says 172.21.15.86:33656) 2024-03-22T19:18:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4b1e00700 1 -- 172.21.15.86:0/1992716492 learned_addr learned my addr 172.21.15.86:0/1992716492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.103+0000 7fa4b1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59252/0 (socket says 172.21.15.86:59252) 2024-03-22T19:18:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b2800700 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693be53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b2800700 1 -- 172.21.15.86:0/1992716492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 msgr2=0x55f693c036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b2800700 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693c036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b2800700 1 -- 172.21.15.86:0/1992716492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 msgr2=0x55f693bf1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b2800700 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b2800700 1 -- 172.21.15.86:0/1992716492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f693af8ea0 con 0x55f693be2420 2024-03-22T19:18:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.104+0000 7fa4b1400700 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:18:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.107+0000 7fa4b2800700 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693be53e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8000f80 tx=0x7fa4a80094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d1958df61fb0d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.111+0000 7fa49ba00700 1 -- 172.21.15.86:0/1992716492 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8008030 con 0x55f693be2420 2024-03-22T19:18:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.111+0000 7fa49ba00700 1 -- 172.21.15.86:0/1992716492 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4a8008190 con 0x55f693be2420 2024-03-22T19:18:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.111+0000 7fa49ba00700 1 -- 172.21.15.86:0/1992716492 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a801bdd0 con 0x55f693be2420 2024-03-22T19:18:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.112+0000 7fa4c828bb80 1 -- 172.21.15.86:0/1992716492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 msgr2=0x55f693be53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.112+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693be53e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8000f80 tx=0x7fa4a80094a0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.113+0000 7fa4c828bb80 1 -- 172.21.15.86:0/1992716492 shutdown_connections 2024-03-22T19:18:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.113+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be2420 0x55f693be53e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.113+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be0c20 0x55f693c036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.113+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/1992716492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.113+0000 7fa4c828bb80 1 -- 172.21.15.86:0/1992716492 >> 172.21.15.86:0/1992716492 conn(0x55f693b56220 msgr2=0x55f693bd7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.116+0000 7fa4c828bb80 1 -- 172.21.15.86:0/1992716492 shutdown_connections 2024-03-22T19:18:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.116+0000 7fa4c828bb80 1 -- 172.21.15.86:0/1992716492 wait complete. 2024-03-22T19:18:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.117+0000 7fa4c828bb80 1 -- start start 2024-03-22T19:18:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.117+0000 7fa4c828bb80 1 Processor -- start 2024-03-22T19:18:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.118+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be0c20 0x55f693bf1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.118+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.118+0000 7fa4c828bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 0x55f693bf0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693bf0d70 con 0x55f693be1580 2024-03-22T19:18:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4c828bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693bf0ee0 con 0x55f693be2420 2024-03-22T19:18:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 0x55f693bf0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 0x55f693bf0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33672/0 (socket says 172.21.15.86:33672) 2024-03-22T19:18:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 -- 172.21.15.86:0/960750735 learned_addr learned my addr 172.21.15.86:0/960750735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f693bf10a0 con 0x55f693be0c20 2024-03-22T19:18:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4c828bb80 10 client.0 _ll_register_callbacks cb 0x7fa49c09a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 -- 172.21.15.86:0/960750735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be0c20 msgr2=0x55f693bf1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be0c20 0x55f693bf1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 -- 172.21.15.86:0/960750735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 msgr2=0x55f693bf02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.119+0000 7fa4b2800700 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.120+0000 7fa4b2800700 1 -- 172.21.15.86:0/960750735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f693960b60 con 0x55f693be2420 2024-03-22T19:18:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.120+0000 7fa48f400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.120+0000 7fa4b2800700 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 0x55f693bf0830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8028040 tx=0x7fa4a80099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f3d8cb7870352c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.121+0000 7fa48f400700 1 -- 172.21.15.86:0/960750735 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8004a30 con 0x55f693be2420 2024-03-22T19:18:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.121+0000 7fa48f400700 1 -- 172.21.15.86:0/960750735 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa4a8021070 con 0x55f693be2420 2024-03-22T19:18:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.121+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4a800f040 con 0x55f693be2420 2024-03-22T19:18:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.122+0000 7fa48f400700 1 -- 172.21.15.86:0/960750735 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4a8004b90 con 0x55f693be2420 2024-03-22T19:18:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.122+0000 7fa4c828bb80 10 client.26309 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.122+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f6939b43b0 con 0x55f693be2420 2024-03-22T19:18:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.127+0000 7fa48f400700 1 -- 172.21.15.86:0/960750735 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4a800f040 con 0x55f693be2420 2024-03-22T19:18:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.127+0000 7fa48f400700 1 -- 172.21.15.86:0/960750735 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa4a8039450 con 0x55f693be2420 2024-03-22T19:18:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.129+0000 7fa4c828bb80 10 client.26309 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.129+0000 7fa4c828bb80 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.129+0000 7fa4c828bb80 20 client.26309 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.129+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c5a2b0 0x55f693c5c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.129+0000 7fa4c828bb80 4 client.26309 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:18:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.129+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f693981cf0 con 0x55f693c5a2b0 2024-03-22T19:18:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.131+0000 7fa4b1e00700 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c5a2b0 0x55f693c5c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.144+0000 7fa4b1e00700 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c5a2b0 0x55f693c5c6d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aaab9c9b26d8667 server_cookie=d6d027749c5f7202 in_seq=0 out_seq=0 2024-03-22T19:18:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.144+0000 7fa48f400700 10 client.26309 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:45.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa48f400700 1 -- 172.21.15.86:0/960750735 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f693981cf0 con 0x55f693c5a2b0 2024-03-22T19:18:45.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa48f400700 10 client.26309 handle_command_reply: tid=0 2024-03-22T19:18:45.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 client.26309 shutdown 2024-03-22T19:18:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 msgr2=0x55f693bf0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 0x55f693bf0830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8028040 tx=0x7fa4a80099a0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 shutdown_connections 2024-03-22T19:18:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x55f693c5a2b0 0x55f693c5c6d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f693be2420 0x55f693bf0830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f693be0c20 0x55f693bf1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 --2- 172.21.15.86:0/960750735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693be1580 0x55f693bf02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.151+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 >> 172.21.15.86:0/960750735 conn(0x55f693b56220 msgr2=0x55f693be53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.154+0000 7fa48f400700 0 client.26309 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:45.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.154+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 shutdown_connections 2024-03-22T19:18:45.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.154+0000 7fa4c828bb80 1 -- 172.21.15.86:0/960750735 wait complete. 2024-03-22T19:18:45.155 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:18:45.156 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:18:45.156 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:18:45.156 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:18:45.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.154+0000 7fa4c828bb80 20 client.26309 trim_cache size 0 max 16384 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.155+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4940a0b90 msgr2=0x7fa4940a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.155+0000 7fa4b4800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4940a0b90 0x7fa4940a3050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009fd0 tx=0x7fa4a8009310 comp rx=0 tx=0).stop 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.155+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 msgr2=0x7fa4ac1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.155+0000 7fa4b4800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 0x7fa4ac1103c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0098e0 tx=0x7fa49c0057b0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.156+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 shutdown_connections 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.156+0000 7fa4b4800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa4940a0b90 0x7fa4940a3050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.156+0000 7fa4b4800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ac108c10 0x7fa4ac1ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.156+0000 7fa4b4800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0737b0 0x7fa4ac10fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.156+0000 7fa4b4800700 1 --2- 172.21.15.86:0/2449432799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ac074d90 0x7fa4ac1103c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.156+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 >> 172.21.15.86:0/2449432799 conn(0x7fa4ac0fc2f0 msgr2=0x7fa4ac102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.157+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 shutdown_connections 2024-03-22T19:18:45.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.157+0000 7fa4b4800700 1 -- 172.21.15.86:0/2449432799 wait complete. 2024-03-22T19:18:45.199 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:18:45.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:18:45.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 -- 172.21.15.86:0/4016082693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58074d90 msgr2=0x7fcb580731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 --2- 172.21.15.86:0/4016082693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58074d90 0x7fcb580731e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b0a0 tx=0x7fcb5000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 -- 172.21.15.86:0/4016082693 shutdown_connections 2024-03-22T19:18:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 --2- 172.21.15.86:0/4016082693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb58108b10 0x7fcb5810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 --2- 172.21.15.86:0/4016082693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58074d90 0x7fcb580731e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 --2- 172.21.15.86:0/4016082693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580737b0 0x7fcb58073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.619+0000 7fcb5d800700 1 -- 172.21.15.86:0/4016082693 >> 172.21.15.86:0/4016082693 conn(0x7fcb580fc1b0 msgr2=0x7fcb580fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.620+0000 7fcb5d800700 1 -- 172.21.15.86:0/4016082693 shutdown_connections 2024-03-22T19:18:45.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.620+0000 7fcb5d800700 1 -- 172.21.15.86:0/4016082693 wait complete. 2024-03-22T19:18:45.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.622+0000 7fcb5d800700 1 Processor -- start 2024-03-22T19:18:45.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.623+0000 7fcb5d800700 1 -- start start 2024-03-22T19:18:45.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.624+0000 7fcb5d800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580737b0 0x7fcb5806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.625+0000 7fcb5d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb58074d90 0x7fcb5806fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.625+0000 7fcb5d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 0x7fcb581ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.625+0000 7fcb5d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580703a0 con 0x7fcb58074d90 2024-03-22T19:18:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.625+0000 7fcb5d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58104670 con 0x7fcb58108b10 2024-03-22T19:18:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.625+0000 7fcb5d800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58104830 con 0x7fcb580737b0 2024-03-22T19:18:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.626+0000 7fcb57400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 0x7fcb581ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.626+0000 7fcb57400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 0x7fcb581ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33682/0 (socket says 172.21.15.86:33682) 2024-03-22T19:18:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.626+0000 7fcb57400700 1 -- 172.21.15.86:0/394579560 learned_addr learned my addr 172.21.15.86:0/394579560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.626+0000 7fcb56000700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb58074d90 0x7fcb5806fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.627+0000 7fcb57400700 1 -- 172.21.15.86:0/394579560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580737b0 msgr2=0x7fcb5806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.627+0000 7fcb57400700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580737b0 0x7fcb5806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.627+0000 7fcb57400700 1 -- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb58074d90 msgr2=0x7fcb5806fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:45.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.627+0000 7fcb57400700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb58074d90 0x7fcb5806fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:45.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.627+0000 7fcb57400700 1 -- 172.21.15.86:0/394579560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50009d00 con 0x7fcb58108b10 2024-03-22T19:18:45.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.629+0000 7fcb57400700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 0x7fcb581ad6f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000d980 tx=0x7fcb4000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.630+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40009940 con 0x7fcb58108b10 2024-03-22T19:18:45.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.630+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40004830 con 0x7fcb58108b10 2024-03-22T19:18:45.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.630+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400054e0 con 0x7fcb58108b10 2024-03-22T19:18:45.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.631+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb581adc30 con 0x7fcb58108b10 2024-03-22T19:18:45.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.631+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581ae0f0 con 0x7fcb58108b10 2024-03-22T19:18:45.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.634+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb58107920 con 0x7fcb58108b10 2024-03-22T19:18:45.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.641+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fcb40004ca0 con 0x7fcb58108b10 2024-03-22T19:18:45.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.644+0000 7fcb4f400700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcb3c0a0b90 0x7fcb3c0a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:45.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.644+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fcb400cc050 con 0x7fcb58108b10 2024-03-22T19:18:45.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.645+0000 7fcb56a00700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcb3c0a0b90 0x7fcb3c0a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:45.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.646+0000 7fcb56a00700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcb3c0a0b90 0x7fcb3c0a3050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b070 tx=0x7fcb50006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:45.654+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb40095a90 con 0x7fcb58108b10 2024-03-22T19:18:45.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:45 smithi086 ceph-mon[38032]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 80 MiB/s wr, 12.80k op/s 2024-03-22T19:18:45.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:45 smithi167 ceph-mon[47728]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 80 MiB/s wr, 12.80k op/s 2024-03-22T19:18:45.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:45 smithi131 ceph-mon[78559]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 80 MiB/s wr, 12.80k op/s 2024-03-22T19:18:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.170+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb58067e30 con 0x7fcb58108b10 2024-03-22T19:18:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.171+0000 7fcb4f400700 1 -- 172.21.15.86:0/394579560 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10924 (secure 0 0 0) 0x7fcb40017030 con 0x7fcb58108b10 2024-03-22T19:18:46.173 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:18:46.173 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:17:14.447772+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:18:46.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.176+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcb3c0a0b90 msgr2=0x7fcb3c0a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.176+0000 7fcb5d800700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcb3c0a0b90 0x7fcb3c0a3050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b070 tx=0x7fcb50006000 comp rx=0 tx=0).stop 2024-03-22T19:18:46.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.176+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 msgr2=0x7fcb581ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.176+0000 7fcb5d800700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 0x7fcb581ad6f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000d980 tx=0x7fcb4000dd40 comp rx=0 tx=0).stop 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.177+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 shutdown_connections 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.177+0000 7fcb5d800700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fcb3c0a0b90 0x7fcb3c0a3050 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b070 tx=0x7fcb50006000 comp rx=0 tx=0).stop 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.177+0000 7fcb5d800700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58108b10 0x7fcb581ad6f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.177+0000 7fcb5d800700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580737b0 0x7fcb5806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.177+0000 7fcb5d800700 1 --2- 172.21.15.86:0/394579560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb58074d90 0x7fcb5806fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.177+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 >> 172.21.15.86:0/394579560 conn(0x7fcb580fc1b0 msgr2=0x7fcb58104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.178+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 shutdown_connections 2024-03-22T19:18:46.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.178+0000 7fcb5d800700 1 -- 172.21.15.86:0/394579560 wait complete. 2024-03-22T19:18:46.184 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 42 2024-03-22T19:18:46.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:18:46.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.660+0000 7f9d6ee00700 1 -- 172.21.15.86:0/516169292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d68100d40 msgr2=0x7f9d68101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.660+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/516169292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d68100d40 0x7f9d68101160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9d580099c0 tx=0x7f9d58009cd0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 -- 172.21.15.86:0/516169292 shutdown_connections 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/516169292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d681028d0 0x7f9d68109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/516169292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d68100d40 0x7f9d68101160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/516169292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68101f30 0x7f9d68102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 -- 172.21.15.86:0/516169292 >> 172.21.15.86:0/516169292 conn(0x7f9d680fc2f0 msgr2=0x7f9d680fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 -- 172.21.15.86:0/516169292 shutdown_connections 2024-03-22T19:18:46.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.661+0000 7f9d6ee00700 1 -- 172.21.15.86:0/516169292 wait complete. 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.664+0000 7f9d6ee00700 1 Processor -- start 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.664+0000 7f9d6ee00700 1 -- start start 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.665+0000 7f9d6ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 0x7f9d68074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.665+0000 7f9d6ee00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d68101f30 0x7f9d680731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.665+0000 7f9d6ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d681028d0 0x7f9d680736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.665+0000 7f9d6ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6819ab50 con 0x7f9d68100d40 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.665+0000 7f9d6ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6819acc0 con 0x7f9d681028d0 2024-03-22T19:18:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.665+0000 7f9d6ee00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6819ae80 con 0x7f9d68101f30 2024-03-22T19:18:46.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 0x7f9d68074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d681028d0 0x7f9d680736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 0x7f9d68074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59324/0 (socket says 172.21.15.86:59324) 2024-03-22T19:18:46.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 -- 172.21.15.86:0/1407053307 learned_addr learned my addr 172.21.15.86:0/1407053307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 -- 172.21.15.86:0/1407053307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d68101f30 msgr2=0x7f9d680731a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T19:18:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d68101f30 0x7f9d680731a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 -- 172.21.15.86:0/1407053307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d681028d0 msgr2=0x7f9d680736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d681028d0 0x7f9d680736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.667+0000 7f9d67e00700 1 -- 172.21.15.86:0/1407053307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580095f0 con 0x7f9d68100d40 2024-03-22T19:18:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.668+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 0x7f9d68074b10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9d58006010 tx=0x7f9d580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.669+0000 7f9d64c00700 1 -- 172.21.15.86:0/1407053307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5801d070 con 0x7f9d68100d40 2024-03-22T19:18:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.669+0000 7f9d64c00700 1 -- 172.21.15.86:0/1407053307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d58004090 con 0x7f9d68100d40 2024-03-22T19:18:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.669+0000 7f9d64c00700 1 -- 172.21.15.86:0/1407053307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58021410 con 0x7f9d68100d40 2024-03-22T19:18:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.669+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d68073c20 con 0x7f9d68100d40 2024-03-22T19:18:46.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.670+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6819b0a0 con 0x7f9d68100d40 2024-03-22T19:18:46.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.675+0000 7f9d64c00700 1 -- 172.21.15.86:0/1407053307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9d5800f5f0 con 0x7f9d68100d40 2024-03-22T19:18:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.677+0000 7f9d828b2b80 1 Processor -- start 2024-03-22T19:18:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.677+0000 7f9d828b2b80 1 -- start start 2024-03-22T19:18:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.678+0000 7f9d64c00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9d4c0a0b10 0x7f9d4c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.679+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9d4c0a0b10 0x7f9d4c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.679+0000 7f9d64c00700 1 -- 172.21.15.86:0/1407053307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9d580cce90 con 0x7f9d68100d40 2024-03-22T19:18:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.680+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53186040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.681+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c53186e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.682+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53186040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.682+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53186040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59326/0 (socket says 172.21.15.86:59326) 2024-03-22T19:18:46.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.682+0000 7f9d67e00700 1 -- 172.21.15.86:0/1660777774 learned_addr learned my addr 172.21.15.86:0/1660777774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:46.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.682+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53187420 0x559c531878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.682+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c530de990 con 0x559c53185c20 2024-03-22T19:18:46.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.683+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c530dc350 con 0x559c53186580 2024-03-22T19:18:46.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.683+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c530d7c40 con 0x559c53187420 2024-03-22T19:18:46.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.683+0000 7f9d67400700 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c53186e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.683+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9d4c0a0b10 0x7f9d4c0a2fd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400ab50 tx=0x7f9d54005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:46.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.684+0000 7f9d67e00700 1 -- 172.21.15.86:0/1660777774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53187420 msgr2=0x559c531878a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:18:46.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.684+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53187420 0x559c531878a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.684+0000 7f9d67e00700 1 -- 172.21.15.86:0/1660777774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 msgr2=0x559c53186e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.684+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c53186e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.684+0000 7f9d67e00700 1 -- 172.21.15.86:0/1660777774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c52f593b0 con 0x559c53185c20 2024-03-22T19:18:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.687+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53186040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9d580a2500 tx=0x7f9d5800f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18503d9216e8f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:46.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.688+0000 7f9d5d600700 1 -- 172.21.15.86:0/1660777774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58099fa0 con 0x559c53185c20 2024-03-22T19:18:46.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.689+0000 7f9d5d600700 1 -- 172.21.15.86:0/1660777774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d5809ad40 con 0x559c53185c20 2024-03-22T19:18:46.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.689+0000 7f9d5d600700 1 -- 172.21.15.86:0/1660777774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5809e170 con 0x559c53185c20 2024-03-22T19:18:46.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 msgr2=0x559c53186040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53186040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9d580a2500 tx=0x7f9d5800f990 comp rx=0 tx=0).stop 2024-03-22T19:18:46.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 shutdown_connections 2024-03-22T19:18:46.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53187420 0x559c531878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53186040 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1660777774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c53186e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.690+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 >> 172.21.15.86:0/1660777774 conn(0x559c530fb220 msgr2=0x559c53180bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.694+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 shutdown_connections 2024-03-22T19:18:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.694+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1660777774 wait complete. 2024-03-22T19:18:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.694+0000 7f9d828b2b80 1 -- start start 2024-03-22T19:18:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.695+0000 7f9d828b2b80 1 Processor -- start 2024-03-22T19:18:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.696+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53185c20 0x559c5318f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.696+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c5318fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.697+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53196c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.697+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c53073d80 con 0x559c53187420 2024-03-22T19:18:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.697+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c53073a20 con 0x559c53186580 2024-03-22T19:18:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.697+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c52f5b6a0 con 0x559c53185c20 2024-03-22T19:18:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.697+0000 7f9d828b2b80 10 client.0 _ll_register_callbacks cb 0x559c52edf3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.698+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.698+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53185c20 0x559c5318f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.698+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53185c20 0x559c5318f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49122/0 (socket says 172.21.15.86:49122) 2024-03-22T19:18:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.698+0000 7f9d67e00700 1 -- 172.21.15.86:0/1168182674 learned_addr learned my addr 172.21.15.86:0/1168182674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.699+0000 7f9d6ce00700 1 -- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53185c20 msgr2=0x559c5318f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.699+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53185c20 0x559c5318f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.699+0000 7f9d6ce00700 1 -- 172.21.15.86:0/1168182674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 msgr2=0x559c5318fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.699+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c5318fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.699+0000 7f9d6ce00700 1 -- 172.21.15.86:0/1168182674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6819b0a0 con 0x559c53187420 2024-03-22T19:18:46.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.701+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53196c30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e630 tx=0x7f9d54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ce5d78b5a7370 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:46.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.701+0000 7f9d52000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:46.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.702+0000 7f9d52000700 1 -- 172.21.15.86:0/1168182674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54004a30 con 0x559c53187420 2024-03-22T19:18:46.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.702+0000 7f9d52000700 1 -- 172.21.15.86:0/1168182674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9d54004b90 con 0x559c53187420 2024-03-22T19:18:46.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.702+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c53178ad0 con 0x559c53187420 2024-03-22T19:18:46.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.703+0000 7f9d52000700 1 -- 172.21.15.86:0/1168182674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d54004d50 con 0x559c53187420 2024-03-22T19:18:46.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.703+0000 7f9d828b2b80 10 client.16188 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.703+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559c52f05b60 con 0x559c53187420 2024-03-22T19:18:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.703+0000 7f9d52000700 1 -- 172.21.15.86:0/1168182674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c53178ad0 con 0x559c53187420 2024-03-22T19:18:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d52000700 1 -- 172.21.15.86:0/1168182674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d5401db30 con 0x559c53187420 2024-03-22T19:18:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d828b2b80 10 client.16188 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d828b2b80 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:18:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d828b2b80 20 client.16188 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c5319b590 0x559c5319d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d828b2b80 4 client.16188 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.704+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c530d0700 con 0x559c5319b590 2024-03-22T19:18:46.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.708+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c5319b590 0x559c5319d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.712+0000 7f9d52000700 10 client.16188 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:18:46.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.712+0000 7f9d67e00700 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c5319b590 0x559c5319d9b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a5bd92c1eafefa6 server_cookie=e19320a6fe409ecb in_seq=0 out_seq=0 2024-03-22T19:18:46.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d52000700 1 -- 172.21.15.86:0/1168182674 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559c530d0700 con 0x559c5319b590 2024-03-22T19:18:46.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d52000700 10 client.16188 handle_command_reply: tid=0 2024-03-22T19:18:46.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d828b2b80 1 client.16188 shutdown 2024-03-22T19:18:46.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 msgr2=0x559c53196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53196c30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e630 tx=0x7f9d54004750 comp rx=0 tx=0).stop 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 shutdown_connections 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53196c30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.718+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c5319b590 0x559c5319d9b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.719+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53185c20 0x559c5318f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.719+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/1168182674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53186580 0x559c5318fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.719+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 >> 172.21.15.86:0/1168182674 conn(0x559c530fb220 msgr2=0x559c531801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.719+0000 7f9d52000700 0 client.16188 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:18:46.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.719+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 shutdown_connections 2024-03-22T19:18:46.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.719+0000 7f9d828b2b80 1 -- 172.21.15.86:0/1168182674 wait complete. 2024-03-22T19:18:46.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.720+0000 7f9d828b2b80 20 client.16188 trim_cache size 0 max 16384 2024-03-22T19:18:46.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:46 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/394579560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:18:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.759+0000 7f9d828b2b80 1 Processor -- start 2024-03-22T19:18:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.759+0000 7f9d828b2b80 1 -- start start 2024-03-22T19:18:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.760+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53185c20 0x559c531a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.760+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c53196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.760+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53187800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.760+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c531872a0 con 0x559c53187420 2024-03-22T19:18:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.760+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5317b210 con 0x559c53185c20 2024-03-22T19:18:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.760+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5318f220 con 0x559c53186580 2024-03-22T19:18:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.761+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53187800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.761+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53187800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59346/0 (socket says 172.21.15.86:59346) 2024-03-22T19:18:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.761+0000 7f9d6ce00700 1 -- 172.21.15.86:0/4230494609 learned_addr learned my addr 172.21.15.86:0/4230494609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.761+0000 7f9d67400700 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c53196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.761+0000 7f9d67e00700 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53185c20 0x559c531a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.762+0000 7f9d6ce00700 1 -- 172.21.15.86:0/4230494609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 msgr2=0x559c53196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.762+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c53196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.762+0000 7f9d6ce00700 1 -- 172.21.15.86:0/4230494609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53185c20 msgr2=0x559c531a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.762+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53185c20 0x559c531a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.762+0000 7f9d6ce00700 1 -- 172.21.15.86:0/4230494609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c5309dea0 con 0x559c53187420 2024-03-22T19:18:46.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.762+0000 7f9d6ce00700 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53187800 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9d5401de90 tx=0x7f9d5401dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b959042120db217f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:46.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.763+0000 7f9d5e000700 1 -- 172.21.15.86:0/4230494609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54005230 con 0x559c53187420 2024-03-22T19:18:46.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.763+0000 7f9d5e000700 1 -- 172.21.15.86:0/4230494609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d5401d450 con 0x559c53187420 2024-03-22T19:18:46.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.763+0000 7f9d5e000700 1 -- 172.21.15.86:0/4230494609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54025bd0 con 0x559c53187420 2024-03-22T19:18:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 -- 172.21.15.86:0/4230494609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 msgr2=0x559c53187800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53187800 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9d5401de90 tx=0x7f9d5401dbb0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 -- 172.21.15.86:0/4230494609 shutdown_connections 2024-03-22T19:18:46.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53187420 0x559c53187800 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53185c20 0x559c531a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/4230494609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c53196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.766+0000 7f9d828b2b80 1 -- 172.21.15.86:0/4230494609 >> 172.21.15.86:0/4230494609 conn(0x559c530fb220 msgr2=0x559c5318d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.767+0000 7f9d828b2b80 1 -- 172.21.15.86:0/4230494609 shutdown_connections 2024-03-22T19:18:46.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.767+0000 7f9d828b2b80 1 -- 172.21.15.86:0/4230494609 wait complete. 2024-03-22T19:18:46.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.767+0000 7f9d828b2b80 1 -- start start 2024-03-22T19:18:46.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.770+0000 7f9d828b2b80 1 Processor -- start 2024-03-22T19:18:46.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.770+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53194b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.770+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c531e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.771+0000 7f9d828b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53187420 0x559c531eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c531ebc70 con 0x559c53185c20 2024-03-22T19:18:46.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c531ebde0 con 0x559c53187420 2024-03-22T19:18:46.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d828b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c531ebfa0 con 0x559c53186580 2024-03-22T19:18:46.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d828b2b80 10 client.0 _ll_register_callbacks cb 0x7f9d5400ec40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:46.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d67e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53194b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d67400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c531e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d67400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c531e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49138/0 (socket says 172.21.15.86:49138) 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.772+0000 7f9d67400700 1 -- 172.21.15.86:0/269945766 learned_addr learned my addr 172.21.15.86:0/269945766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.773+0000 7f9d67e00700 1 -- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 msgr2=0x559c531e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.773+0000 7f9d67e00700 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c531e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.773+0000 7f9d67e00700 1 -- 172.21.15.86:0/269945766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53187420 msgr2=0x559c531eb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.773+0000 7f9d67e00700 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53187420 0x559c531eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.773+0000 7f9d67e00700 1 -- 172.21.15.86:0/269945766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c52f05b60 con 0x559c53185c20 2024-03-22T19:18:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.774+0000 7f9d67e00700 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53194b40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9d58096b40 tx=0x7f9d5809a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf93338f53f2bb64 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.774+0000 7f9d52000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.774+0000 7f9d52000700 1 -- 172.21.15.86:0/269945766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d580b8440 con 0x559c53185c20 2024-03-22T19:18:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.775+0000 7f9d52000700 1 -- 172.21.15.86:0/269945766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9d580cae40 con 0x559c53185c20 2024-03-22T19:18:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.775+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d5809ad40 con 0x559c53185c20 2024-03-22T19:18:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.775+0000 7f9d52000700 1 -- 172.21.15.86:0/269945766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5809a640 con 0x559c53185c20 2024-03-22T19:18:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 10 client.16194 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559c52f5dcd0 con 0x559c53185c20 2024-03-22T19:18:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d52000700 1 -- 172.21.15.86:0/269945766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d58081030 con 0x559c53185c20 2024-03-22T19:18:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d52000700 1 -- 172.21.15.86:0/269945766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d580b8740 con 0x559c53185c20 2024-03-22T19:18:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 10 client.16194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:18:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 20 client.16194 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c531ed480 0x559c531ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 4 client.16194 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:18:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.776+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c52f26cf0 con 0x559c531ed480 2024-03-22T19:18:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.778+0000 7f9d67400700 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c531ed480 0x559c531ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.781+0000 7f9d67400700 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c531ed480 0x559c531ef8a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a80e3f64de917a1f server_cookie=c56c21a9e67ed27d in_seq=0 out_seq=0 2024-03-22T19:18:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.781+0000 7f9d52000700 10 client.16194 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:18:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.781+0000 7f9d52000700 1 -- 172.21.15.86:0/269945766 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c52f26cf0 con 0x559c531ed480 2024-03-22T19:18:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.781+0000 7f9d52000700 10 client.16194 handle_command_reply: tid=0 2024-03-22T19:18:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.781+0000 7f9d828b2b80 1 client.16194 shutdown 2024-03-22T19:18:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 msgr2=0x559c53194b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53194b40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9d58096b40 tx=0x7f9d5809a510 comp rx=0 tx=0).stop 2024-03-22T19:18:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 shutdown_connections 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559c53187420 0x559c531eb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c53185c20 0x559c53194b40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x559c531ed480 0x559c531ef8a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 --2- 172.21.15.86:0/269945766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53186580 0x559c531e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.782+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 >> 172.21.15.86:0/269945766 conn(0x559c530fb220 msgr2=0x559c5318a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.786+0000 7f9d52000700 0 client.16194 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.786+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 shutdown_connections 2024-03-22T19:18:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.786+0000 7f9d828b2b80 1 -- 172.21.15.86:0/269945766 wait complete. 2024-03-22T19:18:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.787+0000 7f9d828b2b80 20 client.16194 trim_cache size 0 max 16384 2024-03-22T19:18:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:18:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.788+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9d4c0a0b10 msgr2=0x7f9d4c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.788+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9d4c0a0b10 0x7f9d4c0a2fd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400ab50 tx=0x7f9d54005e30 comp rx=0 tx=0).stop 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.788+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 msgr2=0x7f9d68074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.788+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 0x7f9d68074b10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9d58006010 tx=0x7f9d580057f0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 shutdown_connections 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9d4c0a0b10 0x7f9d4c0a2fd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d681028d0 0x7f9d680736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d68100d40 0x7f9d68074b10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 --2- 172.21.15.86:0/1407053307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d68101f30 0x7f9d680731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 >> 172.21.15.86:0/1407053307 conn(0x7f9d680fc2f0 msgr2=0x7f9d681052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 shutdown_connections 2024-03-22T19:18:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:46.789+0000 7f9d6ee00700 1 -- 172.21.15.86:0/1407053307 wait complete. 2024-03-22T19:18:46.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:18:46.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:46 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/394579560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:18:46.973 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:46 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/394579560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:18:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 -- 172.21.15.86:0/2705657630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0101f40 msgr2=0x7fa8b01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/2705657630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0101f40 0x7fa8b01023a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa898009920 tx=0x7fa898009c30 comp rx=0 tx=0).stop 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 -- 172.21.15.86:0/2705657630 shutdown_connections 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/2705657630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b01028e0 0x7fa8b0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/2705657630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b0100d50 0x7fa8b0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/2705657630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0101f40 0x7fa8b01023a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.227+0000 7fa8b7c00700 1 -- 172.21.15.86:0/2705657630 >> 172.21.15.86:0/2705657630 conn(0x7fa8b00fc2c0 msgr2=0x7fa8b00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.228+0000 7fa8b7c00700 1 -- 172.21.15.86:0/2705657630 shutdown_connections 2024-03-22T19:18:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.228+0000 7fa8b7c00700 1 -- 172.21.15.86:0/2705657630 wait complete. 2024-03-22T19:18:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.230+0000 7fa8b7c00700 1 Processor -- start 2024-03-22T19:18:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.230+0000 7fa8b7c00700 1 -- start start 2024-03-22T19:18:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b7c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0100d50 0x7fa8b0074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b7c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 0x7fa8b00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b7c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028e0 0x7fa8b00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b7c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0073de0 con 0x7fa8b01028e0 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b7c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0073f50 con 0x7fa8b0100d50 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b7c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a0e90 con 0x7fa8b0101f40 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 0x7fa8b00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 0x7fa8b00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49154/0 (socket says 172.21.15.86:49154) 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8afe00700 1 -- 172.21.15.86:0/4244206684 learned_addr learned my addr 172.21.15.86:0/4244206684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.231+0000 7fa8b5c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028e0 0x7fa8b00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.232+0000 7fa8afe00700 1 -- 172.21.15.86:0/4244206684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0100d50 msgr2=0x7fa8b0074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.232+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0100d50 0x7fa8b0074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.232+0000 7fa8afe00700 1 -- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028e0 msgr2=0x7fa8b00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.232+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028e0 0x7fa8b00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.232+0000 7fa8afe00700 1 -- 172.21.15.86:0/4244206684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa898009520 con 0x7fa8b0101f40 2024-03-22T19:18:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.233+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 0x7fa8b00731a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa89800fc00 tx=0x7fa89800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.234+0000 7fa8ad600700 1 -- 172.21.15.86:0/4244206684 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89801c070 con 0x7fa8b0101f40 2024-03-22T19:18:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.235+0000 7fa8ad600700 1 -- 172.21.15.86:0/4244206684 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa89800ba60 con 0x7fa8b0101f40 2024-03-22T19:18:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.235+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b01a1090 con 0x7fa8b0101f40 2024-03-22T19:18:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.235+0000 7fa8ad600700 1 -- 172.21.15.86:0/4244206684 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898004bf0 con 0x7fa8b0101f40 2024-03-22T19:18:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.235+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b01a1460 con 0x7fa8b0101f40 2024-03-22T19:18:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.240+0000 7fa8ad600700 1 -- 172.21.15.86:0/4244206684 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa898020070 con 0x7fa8b0101f40 2024-03-22T19:18:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.245+0000 7fa8ad600700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8940a0b60 0x7fa8940a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.245+0000 7fa8cb72db80 1 Processor -- start 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.245+0000 7fa8ad600700 1 -- 172.21.15.86:0/4244206684 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8980cce90 con 0x7fa8b0101f40 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.245+0000 7fa8cb72db80 1 -- start start 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.246+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.246+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.246+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.246+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a532990 con 0x55fd1a5e3c00 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.246+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a530350 con 0x55fd1a5e2370 2024-03-22T19:18:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.246+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a52bc40 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.248+0000 7fa8b5200700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8940a0b60 0x7fa8940a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.248+0000 7fa8b5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.248+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.248+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49158/0 (socket says 172.21.15.86:49158) 2024-03-22T19:18:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.248+0000 7fa8b5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5e2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33776/0 (socket says 172.21.15.86:33776) 2024-03-22T19:18:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.248+0000 7fa8afe00700 1 -- 172.21.15.86:0/3095109086 learned_addr learned my addr 172.21.15.86:0/3095109086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5c00700 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5200700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8940a0b60 0x7fa8940a3020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0101da0 tx=0x7fa8a80123f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5c00700 1 -- 172.21.15.86:0/3095109086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e2cd0 msgr2=0x55fd1a5e3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5c00700 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5e3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5c00700 1 -- 172.21.15.86:0/3095109086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 msgr2=0x55fd1a5e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5c00700 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5e2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.249+0000 7fa8b5c00700 1 -- 172.21.15.86:0/3095109086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd1a3ad3b0 con 0x55fd1a5e3c00 2024-03-22T19:18:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.250+0000 7fa8b5c00700 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5e4090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009db0 tx=0x7fa8a0009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a275793d1318044 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.251+0000 7fa8a6000700 1 -- 172.21.15.86:0/3095109086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0011070 con 0x55fd1a5e3c00 2024-03-22T19:18:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.251+0000 7fa8a6000700 1 -- 172.21.15.86:0/3095109086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8a00098e0 con 0x55fd1a5e3c00 2024-03-22T19:18:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.251+0000 7fa8a6000700 1 -- 172.21.15.86:0/3095109086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0015d50 con 0x55fd1a5e3c00 2024-03-22T19:18:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.253+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3095109086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e3c00 msgr2=0x55fd1a5e4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.253+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5e4090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009db0 tx=0x7fa8a0009de0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3095109086 shutdown_connections 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5e4090 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3095109086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3095109086 >> 172.21.15.86:0/3095109086 conn(0x55fd1a54f220 msgr2=0x55fd1a5d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3095109086 shutdown_connections 2024-03-22T19:18:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.254+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3095109086 wait complete. 2024-03-22T19:18:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.255+0000 7fa8cb72db80 1 -- start start 2024-03-22T19:18:47.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.257+0000 7fa8cb72db80 1 Processor -- start 2024-03-22T19:18:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.258+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.258+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.258+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.258+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a4c7d80 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.258+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a4c7a20 con 0x55fd1a5e2370 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.258+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a3af6a0 con 0x55fd1a5e3c00 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5ec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59388/0 (socket says 172.21.15.86:59388) 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 -- 172.21.15.86:0/3435440377 learned_addr learned my addr 172.21.15.86:0/3435440377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8b5c00700 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 -- 172.21.15.86:0/3435440377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 msgr2=0x55fd1a5f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8cb72db80 10 client.0 _ll_register_callbacks cb 0x55fd1a3333a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 -- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 msgr2=0x55fd1a5ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.259+0000 7fa8afe00700 1 -- 172.21.15.86:0/3435440377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a0009a60 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.260+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd1a524700 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.260+0000 7fa8afe00700 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5ec810 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa8980caa70 tx=0x7fa898099fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b491cd7bce4ea041 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.260+0000 7fa89ea00700 10 client.16206 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.260+0000 7fa89ea00700 1 -- 172.21.15.86:0/3435440377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89809a640 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.261+0000 7fa89ea00700 1 -- 172.21.15.86:0/3435440377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8980cae40 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.261+0000 7fa89ea00700 1 -- 172.21.15.86:0/3435440377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa89809e470 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.261+0000 7fa8cb72db80 10 client.16206 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.261+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=42}) v3 -- 0x55fd1a359b60 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa89ea00700 1 -- 172.21.15.86:0/3435440377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa89809e120 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa8cb72db80 10 client.16206 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa8cb72db80 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:18:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa8cb72db80 20 client.16206 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5f7ac0 0x55fd1a5f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa8cb72db80 4 client.16206 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.262+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd1a37acf0 con 0x55fd1a5f7ac0 2024-03-22T19:18:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.263+0000 7fa8b5200700 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5f7ac0 0x55fd1a5f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.270+0000 7fa8b5200700 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5f7ac0 0x55fd1a5f9ee0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=537f55e64bc52460 server_cookie=d9394d7d1d3b0e16 in_seq=0 out_seq=0 2024-03-22T19:18:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.270+0000 7fa89ea00700 10 client.16206 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:18:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.273+0000 7fa89ea00700 1 -- 172.21.15.86:0/3435440377 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fd1a37acf0 con 0x55fd1a5f7ac0 2024-03-22T19:18:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.273+0000 7fa89ea00700 10 client.16206 handle_command_reply: tid=0 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.273+0000 7fa8cb72db80 1 client.16206 shutdown 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 msgr2=0x55fd1a5ec810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5ec810 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa8980caa70 tx=0x7fa898099fa0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 shutdown_connections 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5f7ac0 0x55fd1a5f9ee0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2370 0x55fd1a5ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/3435440377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5ec810 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 >> 172.21.15.86:0/3435440377 conn(0x55fd1a54f220 msgr2=0x55fd1a5e1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa89ea00700 0 client.16206 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:18:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 shutdown_connections 2024-03-22T19:18:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.274+0000 7fa8cb72db80 1 -- 172.21.15.86:0/3435440377 wait complete. 2024-03-22T19:18:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.275+0000 7fa8cb72db80 20 client.16206 trim_cache size 0 max 16384 2024-03-22T19:18:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.303+0000 7fa8cb72db80 1 Processor -- start 2024-03-22T19:18:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.304+0000 7fa8cb72db80 1 -- start start 2024-03-22T19:18:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.305+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.305+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.305+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.305+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.305+0000 7fa8afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33802/0 (socket says 172.21.15.86:33802) 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.305+0000 7fa8afe00700 1 -- 172.21.15.86:0/4032217423 learned_addr learned my addr 172.21.15.86:0/4032217423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a5ead20 con 0x55fd1a5e2370 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a5e9df0 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a5e76e0 con 0x55fd1a5e3c00 2024-03-22T19:18:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8b5200700 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8afe00700 1 -- 172.21.15.86:0/4032217423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 msgr2=0x55fd1a5f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8afe00700 1 -- 172.21.15.86:0/4032217423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 msgr2=0x55fd1a5fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5fe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.306+0000 7fa8afe00700 1 -- 172.21.15.86:0/4032217423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd1a4f1ea0 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.307+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5f3810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa89809a5d0 tx=0x7fa898019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3711c66c03db007 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.313+0000 7fa8a6a00700 1 -- 172.21.15.86:0/4032217423 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980cae40 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.313+0000 7fa8a6a00700 1 -- 172.21.15.86:0/4032217423 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd1a524700 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.313+0000 7fa8a6a00700 1 -- 172.21.15.86:0/4032217423 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980a25a0 con 0x55fd1a5e2cd0 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.314+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 msgr2=0x55fd1a5f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.314+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5f3810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa89809a5d0 tx=0x7fa898019000 comp rx=0 tx=0).stop 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.315+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 shutdown_connections 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.315+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.315+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.315+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4032217423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5e2cd0 0x55fd1a5f3810 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa89809a5d0 tx=0x7fa898019000 comp rx=0 tx=0).stop 2024-03-22T19:18:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.315+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 >> 172.21.15.86:0/4032217423 conn(0x55fd1a54f220 msgr2=0x55fd1a5ea170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.315+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 shutdown_connections 2024-03-22T19:18:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.316+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4032217423 wait complete. 2024-03-22T19:18:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.316+0000 7fa8cb72db80 1 -- start start 2024-03-22T19:18:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.316+0000 7fa8cb72db80 1 Processor -- start 2024-03-22T19:18:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.318+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8cb72db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5efbd0 0x55fd1a5f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a5f05e0 con 0x55fd1a5e2370 2024-03-22T19:18:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a5f0750 con 0x55fd1a5efbd0 2024-03-22T19:18:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8cb72db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1a5da920 con 0x55fd1a5e3c00 2024-03-22T19:18:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8cb72db80 10 client.0 _ll_register_callbacks cb 0x55fd1a3333a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.319+0000 7fa8b5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5efbd0 0x55fd1a5f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5efbd0 0x55fd1a5f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33810/0 (socket says 172.21.15.86:33810) 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59404/0 (socket says 172.21.15.86:59404) 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5c00700 1 -- 172.21.15.86:0/4292986709 learned_addr learned my addr 172.21.15.86:0/4292986709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 -- 172.21.15.86:0/4292986709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 msgr2=0x55fd1a5f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 -- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5efbd0 msgr2=0x55fd1a5f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5efbd0 0x55fd1a5f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.320+0000 7fa8b5200700 1 -- 172.21.15.86:0/4292986709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd1a359b60 con 0x55fd1a5e2370 2024-03-22T19:18:47.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.322+0000 7fa8b5200700 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5f30e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55fd1a5f7ea0 tx=0x7fa8a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5e6e1b39a55e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.322+0000 7fa89ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:47.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.324+0000 7fa89ea00700 1 -- 172.21.15.86:0/4292986709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8010e20 con 0x55fd1a5e2370 2024-03-22T19:18:47.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.324+0000 7fa89ea00700 1 -- 172.21.15.86:0/4292986709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa8a8024050 con 0x55fd1a5e2370 2024-03-22T19:18:47.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.324+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd1a5ccad0 con 0x55fd1a5e2370 2024-03-22T19:18:47.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.324+0000 7fa89ea00700 1 -- 172.21.15.86:0/4292986709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a8026690 con 0x55fd1a5e2370 2024-03-22T19:18:47.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.324+0000 7fa8cb72db80 10 client.16212 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:47.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.324+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55fd1a3b1cd0 con 0x55fd1a5e2370 2024-03-22T19:18:47.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa89ea00700 1 -- 172.21.15.86:0/4292986709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd1a5ccad0 con 0x55fd1a5e2370 2024-03-22T19:18:47.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa89ea00700 1 -- 172.21.15.86:0/4292986709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8a802f8c0 con 0x55fd1a5e2370 2024-03-22T19:18:47.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa8cb72db80 10 client.16212 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:47.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa8cb72db80 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:18:47.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa8cb72db80 20 client.16212 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:47.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5dc240 0x55fd1a5de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa8cb72db80 4 client.16212 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:18:47.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.325+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd1a3afce0 con 0x55fd1a5dc240 2024-03-22T19:18:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.328+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5dc240 0x55fd1a5de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.332+0000 7fa8afe00700 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5dc240 0x55fd1a5de630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=520c4e08837e42b7 server_cookie=1e129102b192d45e in_seq=0 out_seq=0 2024-03-22T19:18:47.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.332+0000 7fa89ea00700 10 client.16212 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:18:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.337+0000 7fa89ea00700 1 -- 172.21.15.86:0/4292986709 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd1a3afce0 con 0x55fd1a5dc240 2024-03-22T19:18:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.337+0000 7fa89ea00700 10 client.16212 handle_command_reply: tid=0 2024-03-22T19:18:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.337+0000 7fa8cb72db80 1 client.16212 shutdown 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 msgr2=0x55fd1a5f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5f30e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55fd1a5f7ea0 tx=0x7fa8a801a000 comp rx=0 tx=0).stop 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 shutdown_connections 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd1a5efbd0 0x55fd1a5f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd1a5e2370 0x55fd1a5f30e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55fd1a5dc240 0x55fd1a5de630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 --2- 172.21.15.86:0/4292986709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fd1a5e3c00 0x55fd1a5f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 >> 172.21.15.86:0/4292986709 conn(0x55fd1a54f220 msgr2=0x55fd1a5e6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.338+0000 7fa89ea00700 0 client.16212 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:18:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.339+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 shutdown_connections 2024-03-22T19:18:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.339+0000 7fa8cb72db80 1 -- 172.21.15.86:0/4292986709 wait complete. 2024-03-22T19:18:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.340+0000 7fa8cb72db80 20 client.16212 trim_cache size 0 max 16384 2024-03-22T19:18:47.341 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:18:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.342+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8940a0b60 msgr2=0x7fa8940a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.342+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8940a0b60 0x7fa8940a3020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0101da0 tx=0x7fa8a80123f0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.342+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 msgr2=0x7fa8b00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.342+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 0x7fa8b00731a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa89800fc00 tx=0x7fa89800b840 comp rx=0 tx=0).stop 2024-03-22T19:18:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 shutdown_connections 2024-03-22T19:18:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa8940a0b60 0x7fa8940a3020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028e0 0x7fa8b00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8b0100d50 0x7fa8b0074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 --2- 172.21.15.86:0/4244206684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0101f40 0x7fa8b00731a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 >> 172.21.15.86:0/4244206684 conn(0x7fa8b00fc2c0 msgr2=0x7fa8b0105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 shutdown_connections 2024-03-22T19:18:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.343+0000 7fa8b7c00700 1 -- 172.21.15.86:0/4244206684 wait complete. 2024-03-22T19:18:47.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:18:47.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:47 smithi086 ceph-mon[38032]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 83 MiB/s wr, 13.30k op/s 2024-03-22T19:18:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 -- 172.21.15.86:0/2814574752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc1028d0 msgr2=0x7fe3fc109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 --2- 172.21.15.86:0/2814574752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc109180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400b0d0 tx=0x7fe3f400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 -- 172.21.15.86:0/2814574752 shutdown_connections 2024-03-22T19:18:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 --2- 172.21.15.86:0/2814574752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc109180 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 --2- 172.21.15.86:0/2814574752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc100d40 0x7fe3fc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 --2- 172.21.15.86:0/2814574752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc101f30 0x7fe3fc102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 -- 172.21.15.86:0/2814574752 >> 172.21.15.86:0/2814574752 conn(0x7fe3fc0fc2f0 msgr2=0x7fe3fc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.768+0000 7fe400a00700 1 -- 172.21.15.86:0/2814574752 shutdown_connections 2024-03-22T19:18:47.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.769+0000 7fe400a00700 1 -- 172.21.15.86:0/2814574752 wait complete. 2024-03-22T19:18:47.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.770+0000 7fe400a00700 1 Processor -- start 2024-03-22T19:18:47.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.770+0000 7fe400a00700 1 -- start start 2024-03-22T19:18:47.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe400a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc100d40 0x7fe3fc19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe400a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc101f30 0x7fe3fc19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe400a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc1a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe400a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc1a4fb0 con 0x7fe3fc100d40 2024-03-22T19:18:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe400a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc191ea0 con 0x7fe3fc101f30 2024-03-22T19:18:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe400a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc192030 con 0x7fe3fc1028d0 2024-03-22T19:18:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe3f9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc100d40 0x7fe3fc19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc1a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc1a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49210/0 (socket says 172.21.15.86:49210) 2024-03-22T19:18:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.771+0000 7fe3fa800700 1 -- 172.21.15.86:0/4279689902 learned_addr learned my addr 172.21.15.86:0/4279689902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.772+0000 7fe3f9400700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc101f30 0x7fe3fc19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.773+0000 7fe3f9e00700 1 -- 172.21.15.86:0/4279689902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1028d0 msgr2=0x7fe3fc1a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.773+0000 7fe3f9e00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc1a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.773+0000 7fe3f9e00700 1 -- 172.21.15.86:0/4279689902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc101f30 msgr2=0x7fe3fc19d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.773+0000 7fe3f9e00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc101f30 0x7fe3fc19d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.773+0000 7fe3f9e00700 1 -- 172.21.15.86:0/4279689902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f4009d00 con 0x7fe3fc100d40 2024-03-22T19:18:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.773+0000 7fe3f9e00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc100d40 0x7fe3fc19d480 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc101d90 tx=0x7fe3ec00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.774+0000 7fe3f2a00700 1 -- 172.21.15.86:0/4279689902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011830 con 0x7fe3fc100d40 2024-03-22T19:18:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.774+0000 7fe3f2a00700 1 -- 172.21.15.86:0/4279689902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ec011e50 con 0x7fe3fc100d40 2024-03-22T19:18:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.774+0000 7fe3f2a00700 1 -- 172.21.15.86:0/4279689902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec010e50 con 0x7fe3fc100d40 2024-03-22T19:18:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.774+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc192310 con 0x7fe3fc100d40 2024-03-22T19:18:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.775+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc192770 con 0x7fe3fc100d40 2024-03-22T19:18:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.784+0000 7fe414421b80 1 Processor -- start 2024-03-22T19:18:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.786+0000 7fe414421b80 1 -- start start 2024-03-22T19:18:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe414421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd24d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe414421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd24de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe414421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe414421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd1a5990 con 0x55f5dd24d580 2024-03-22T19:18:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe414421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd1a3350 con 0x55f5dd24cc20 2024-03-22T19:18:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe414421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd19ec40 con 0x55f5dd24e420 2024-03-22T19:18:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd24d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd24d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33848/0 (socket says 172.21.15.86:33848) 2024-03-22T19:18:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.787+0000 7fe3fa800700 1 -- 172.21.15.86:0/2550523415 learned_addr learned my addr 172.21.15.86:0/2550523415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.789+0000 7fe3f9400700 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.790+0000 7fe3fa800700 1 -- 172.21.15.86:0/2550523415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 msgr2=0x55f5dd24e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.790+0000 7fe3fa800700 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.790+0000 7fe3fa800700 1 -- 172.21.15.86:0/2550523415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 msgr2=0x55f5dd24de40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:18:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.790+0000 7fe3fa800700 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd24de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.790+0000 7fe3fa800700 1 -- 172.21.15.86:0/2550523415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5dd0203b0 con 0x55f5dd24cc20 2024-03-22T19:18:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.790+0000 7fe3fa800700 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd24d040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ce80 tx=0x7fe3f400ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8b8f9be0d4b500 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.791+0000 7fe3e7000700 1 -- 172.21.15.86:0/2550523415 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400e070 con 0x55f5dd24cc20 2024-03-22T19:18:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.791+0000 7fe3e7000700 1 -- 172.21.15.86:0/2550523415 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3f4007660 con 0x55f5dd24cc20 2024-03-22T19:18:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.791+0000 7fe3f2a00700 1 -- 172.21.15.86:0/4279689902 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe3ec0104f0 con 0x7fe3fc100d40 2024-03-22T19:18:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.794+0000 7fe3f2a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3e00a0a90 0x7fe3e00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.794+0000 7fe3f2a00700 1 -- 172.21.15.86:0/4279689902 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe3ec0cbac0 con 0x7fe3fc100d40 2024-03-22T19:18:47.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.794+0000 7fe3e7000700 1 -- 172.21.15.86:0/2550523415 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40053e0 con 0x55f5dd24cc20 2024-03-22T19:18:47.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.795+0000 7fe414421b80 1 -- 172.21.15.86:0/2550523415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 msgr2=0x55f5dd24d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.795+0000 7fe414421b80 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd24d040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ce80 tx=0x7fe3f400ceb0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe3f9e00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3e00a0a90 0x7fe3e00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe414421b80 1 -- 172.21.15.86:0/2550523415 shutdown_connections 2024-03-22T19:18:47.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe414421b80 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe414421b80 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd24d040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe414421b80 1 --2- 172.21.15.86:0/2550523415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd24de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe414421b80 1 -- 172.21.15.86:0/2550523415 >> 172.21.15.86:0/2550523415 conn(0x55f5dd1c2220 msgr2=0x55f5dd247bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.796+0000 7fe414421b80 1 -- 172.21.15.86:0/2550523415 shutdown_connections 2024-03-22T19:18:47.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.797+0000 7fe414421b80 1 -- 172.21.15.86:0/2550523415 wait complete. 2024-03-22T19:18:47.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.797+0000 7fe414421b80 1 -- start start 2024-03-22T19:18:47.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.798+0000 7fe414421b80 1 Processor -- start 2024-03-22T19:18:47.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.798+0000 7fe3f9e00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3e00a0a90 0x7fe3e00a2f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0a1090 tx=0x7fe3ec00bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd256a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd256fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd25e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd13ad80 con 0x55f5dd24d580 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd13aa20 con 0x55f5dd24cc20 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd0226a0 con 0x55f5dd24e420 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe414421b80 10 client.0 _ll_register_callbacks cb 0x55f5dcfa63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd256a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd256a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33850/0 (socket says 172.21.15.86:33850) 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.799+0000 7fe3fa800700 1 -- 172.21.15.86:0/2530974691 learned_addr learned my addr 172.21.15.86:0/2530974691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.800+0000 7fe3f9400700 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd256fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3fa800700 1 -- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 msgr2=0x55f5dd25e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3fa800700 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd25e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3fa800700 1 -- 172.21.15.86:0/2530974691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 msgr2=0x55f5dd256fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3fa800700 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd256fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3fa800700 1 -- 172.21.15.86:0/2530974691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc109ad0 con 0x55f5dd24cc20 2024-03-22T19:18:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3fa800700 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd256a80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4004d80 tx=0x7fe3f4004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f067848c76e8ec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.801+0000 7fe3db400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.804+0000 7fe3db400700 1 -- 172.21.15.86:0/2530974691 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400e040 con 0x55f5dd24cc20 2024-03-22T19:18:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.805+0000 7fe3db400700 1 -- 172.21.15.86:0/2530974691 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe3f40077d0 con 0x55f5dd24cc20 2024-03-22T19:18:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.805+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5dd23fad0 con 0x55f5dd24cc20 2024-03-22T19:18:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.805+0000 7fe3db400700 1 -- 172.21.15.86:0/2530974691 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f4005540 con 0x55f5dd24cc20 2024-03-22T19:18:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.805+0000 7fe414421b80 10 client.26345 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.805+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f5dcfccb60 con 0x55f5dd24cc20 2024-03-22T19:18:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.810+0000 7fe3db400700 1 -- 172.21.15.86:0/2530974691 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5dd23fad0 con 0x55f5dd24cc20 2024-03-22T19:18:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.810+0000 7fe3db400700 1 -- 172.21.15.86:0/2530974691 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3f400b100 con 0x55f5dd24cc20 2024-03-22T19:18:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.810+0000 7fe414421b80 10 client.26345 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.810+0000 7fe414421b80 10 client.26345 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.811+0000 7fe414421b80 20 client.26345 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.811+0000 7fe414421b80 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd262420 0x55f5dd264840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.811+0000 7fe414421b80 4 client.26345 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.811+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5dd197700 con 0x55f5dd262420 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.811+0000 7fe3f9400700 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd262420 0x55f5dd264840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.812+0000 7fe3db400700 10 client.26345 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:18:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.814+0000 7fe3f9400700 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd262420 0x55f5dd264840 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ff303f1f745528f server_cookie=babdda2c2188a31b in_seq=0 out_seq=0 2024-03-22T19:18:47.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.817+0000 7fe3db400700 1 -- 172.21.15.86:0/2530974691 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f5dd197700 con 0x55f5dd262420 2024-03-22T19:18:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.817+0000 7fe3db400700 10 client.26345 handle_command_reply: tid=0 2024-03-22T19:18:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.817+0000 7fe414421b80 1 client.26345 shutdown 2024-03-22T19:18:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 msgr2=0x55f5dd256a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd256a80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4004d80 tx=0x7fe3f4004db0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 shutdown_connections 2024-03-22T19:18:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd262420 0x55f5dd264840 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd25e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe3db400700 0 client.26345 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:18:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd256a80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 --2- 172.21.15.86:0/2530974691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd256fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.818+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 >> 172.21.15.86:0/2530974691 conn(0x55f5dd1c2220 msgr2=0x55f5dd2471f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.820+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 shutdown_connections 2024-03-22T19:18:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.820+0000 7fe414421b80 1 -- 172.21.15.86:0/2530974691 wait complete. 2024-03-22T19:18:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.822+0000 7fe414421b80 20 client.26345 trim_cache size 0 max 16384 2024-03-22T19:18:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:47 smithi167 ceph-mon[47728]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 83 MiB/s wr, 13.30k op/s 2024-03-22T19:18:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.851+0000 7fe414421b80 1 Processor -- start 2024-03-22T19:18:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.851+0000 7fe414421b80 1 -- start start 2024-03-22T19:18:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe414421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd248930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe414421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd25bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe414421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe414421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd257c70 con 0x55f5dd24d580 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe414421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd23f610 con 0x55f5dd24cc20 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe414421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd24dc30 con 0x55f5dd24e420 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd248930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.853+0000 7fe3fa800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd248930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33860/0 (socket says 172.21.15.86:33860) 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.853+0000 7fe3fa800700 1 -- 172.21.15.86:0/1274990458 learned_addr learned my addr 172.21.15.86:0/1274990458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.853+0000 7fe3fa800700 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.852+0000 7fe3f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd25bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.854+0000 7fe3f9400700 1 -- 172.21.15.86:0/1274990458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 msgr2=0x55f5dd24e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.854+0000 7fe3f9400700 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.854+0000 7fe3f9400700 1 -- 172.21.15.86:0/1274990458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 msgr2=0x55f5dd248930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.854+0000 7fe3f9400700 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd248930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.854+0000 7fe3f9400700 1 -- 172.21.15.86:0/1274990458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5dd164ea0 con 0x55f5dd24d580 2024-03-22T19:18:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.854+0000 7fe3f9400700 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd25bfb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55f5dd265150 tx=0x7fe3e8005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40724f861fbdd770 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.855+0000 7fe3e7a00700 1 -- 172.21.15.86:0/1274990458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8016040 con 0x55f5dd24d580 2024-03-22T19:18:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.855+0000 7fe3e7a00700 1 -- 172.21.15.86:0/1274990458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e800ce70 con 0x55f5dd24d580 2024-03-22T19:18:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.855+0000 7fe3e7a00700 1 -- 172.21.15.86:0/1274990458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8005720 con 0x55f5dd24d580 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.856+0000 7fe414421b80 1 -- 172.21.15.86:0/1274990458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 msgr2=0x55f5dd25bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd25bfb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55f5dd265150 tx=0x7fe3e8005f60 comp rx=0 tx=0).stop 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 -- 172.21.15.86:0/1274990458 shutdown_connections 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24e420 0x55f5dd24e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24cc20 0x55f5dd248930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 --2- 172.21.15.86:0/1274990458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24d580 0x55f5dd25bfb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 -- 172.21.15.86:0/1274990458 >> 172.21.15.86:0/1274990458 conn(0x55f5dd1c2220 msgr2=0x55f5dd243a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 -- 172.21.15.86:0/1274990458 shutdown_connections 2024-03-22T19:18:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.857+0000 7fe414421b80 1 -- 172.21.15.86:0/1274990458 wait complete. 2024-03-22T19:18:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.858+0000 7fe414421b80 1 -- start start 2024-03-22T19:18:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.859+0000 7fe414421b80 1 Processor -- start 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.859+0000 7fe414421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24cc20 0x55f5dd2581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe414421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 0x55f5dd2586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe414421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24e420 0x55f5dd25b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe414421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd25bac0 con 0x55f5dd24e420 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe414421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd25bc30 con 0x55f5dd24d580 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe414421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dd25bdf0 con 0x55f5dd24cc20 2024-03-22T19:18:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe414421b80 10 client.0 _ll_register_callbacks cb 0x55f5dcfa63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe3f9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 0x55f5dd2586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe3f9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 0x55f5dd2586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33876/0 (socket says 172.21.15.86:33876) 2024-03-22T19:18:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.860+0000 7fe3f9400700 1 -- 172.21.15.86:0/647406915 learned_addr learned my addr 172.21.15.86:0/647406915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.861+0000 7fe3fa800700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24cc20 0x55f5dd2581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.862+0000 7fe3f9400700 1 -- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24cc20 msgr2=0x55f5dd2581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.862+0000 7fe3fa800700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24e420 0x55f5dd25b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.862+0000 7fe3f9400700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24cc20 0x55f5dd2581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.862+0000 7fe3f9400700 1 -- 172.21.15.86:0/647406915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24e420 msgr2=0x55f5dd25b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.862+0000 7fe3f9400700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24e420 0x55f5dd25b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.863+0000 7fe3f9400700 1 -- 172.21.15.86:0/647406915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e800ca00 con 0x55f5dd24d580 2024-03-22T19:18:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.863+0000 7fe3fa800700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24e420 0x55f5dd25b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:18:47.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.864+0000 7fe3f9400700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 0x55f5dd2586f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8005e20 tx=0x7fe3e8005160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc546721212fc926 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:47.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.864+0000 7fe3db400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.867+0000 7fe3db400700 1 -- 172.21.15.86:0/647406915 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e802a070 con 0x55f5dd24d580 2024-03-22T19:18:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.867+0000 7fe3db400700 1 -- 172.21.15.86:0/647406915 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe3e80049a0 con 0x55f5dd24d580 2024-03-22T19:18:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.867+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3f4007660 con 0x55f5dd24d580 2024-03-22T19:18:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.869+0000 7fe3db400700 1 -- 172.21.15.86:0/647406915 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e8016040 con 0x55f5dd24d580 2024-03-22T19:18:47.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.870+0000 7fe414421b80 10 client.26351 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:47.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.870+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f5dd024cd0 con 0x55f5dd24d580 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.871+0000 7fe3db400700 1 -- 172.21.15.86:0/647406915 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3f4007660 con 0x55f5dd24d580 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.871+0000 7fe3db400700 1 -- 172.21.15.86:0/647406915 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3e8025d10 con 0x55f5dd24d580 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.872+0000 7fe414421b80 10 client.26351 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.872+0000 7fe414421b80 10 client.26351 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.872+0000 7fe414421b80 20 client.26351 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.872+0000 7fe414421b80 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd2c6120 0x55f5dd2c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.872+0000 7fe414421b80 4 client.26351 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:18:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.872+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5dcfedcf0 con 0x55f5dd2c6120 2024-03-22T19:18:47.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.873+0000 7fe3fa800700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd2c6120 0x55f5dd2c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.877+0000 7fe3fa800700 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd2c6120 0x55f5dd2c8540 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=521c0f2f97e30d0d server_cookie=4f4c6459036fe4aa in_seq=0 out_seq=0 2024-03-22T19:18:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.879+0000 7fe3db400700 10 client.26351 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:18:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.880+0000 7fe3db400700 1 -- 172.21.15.86:0/647406915 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5dcfedcf0 con 0x55f5dd2c6120 2024-03-22T19:18:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.880+0000 7fe3db400700 10 client.26351 handle_command_reply: tid=0 2024-03-22T19:18:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.881+0000 7fe414421b80 1 client.26351 shutdown 2024-03-22T19:18:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.884+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 msgr2=0x55f5dd2586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.884+0000 7fe414421b80 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 0x55f5dd2586f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8005e20 tx=0x7fe3e8005160 comp rx=0 tx=0).stop 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 shutdown_connections 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x55f5dd2c6120 0x55f5dd2c8540 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5dd24e420 0x55f5dd25b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5dd24cc20 0x55f5dd2581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 --2- 172.21.15.86:0/647406915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dd24d580 0x55f5dd2586f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe3db400700 0 client.26351 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 >> 172.21.15.86:0/647406915 conn(0x55f5dd1c2220 msgr2=0x55f5dd255fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 shutdown_connections 2024-03-22T19:18:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.886+0000 7fe414421b80 1 -- 172.21.15.86:0/647406915 wait complete. 2024-03-22T19:18:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.887+0000 7fe414421b80 20 client.26351 trim_cache size 0 max 16384 2024-03-22T19:18:47.888 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.887+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3e00a0a90 msgr2=0x7fe3e00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.887+0000 7fe400a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3e00a0a90 0x7fe3e00a2f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0a1090 tx=0x7fe3ec00bf90 comp rx=0 tx=0).stop 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc100d40 msgr2=0x7fe3fc19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc100d40 0x7fe3fc19d480 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc101d90 tx=0x7fe3ec00b7e0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 shutdown_connections 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe3e00a0a90 0x7fe3e00a2f50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1028d0 0x7fe3fc1a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc100d40 0x7fe3fc19d480 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 --2- 172.21.15.86:0/4279689902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc101f30 0x7fe3fc19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:47.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 >> 172.21.15.86:0/4279689902 conn(0x7fe3fc0fc2f0 msgr2=0x7fe3fc103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:47.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 shutdown_connections 2024-03-22T19:18:47.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:47.888+0000 7fe400a00700 1 -- 172.21.15.86:0/4279689902 wait complete. 2024-03-22T19:18:47.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:18:47.976 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:47 smithi131 ceph-mon[78559]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 83 MiB/s wr, 13.30k op/s 2024-03-22T19:18:48.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.314+0000 7f0f3d200700 1 -- 172.21.15.86:0/1687952347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 msgr2=0x7f0f38111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.314+0000 7f0f3d200700 1 --2- 172.21.15.86:0/1687952347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 0x7f0f38111580 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009f90 tx=0x7f0f3000b330 comp rx=0 tx=0).stop 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 -- 172.21.15.86:0/1687952347 shutdown_connections 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 --2- 172.21.15.86:0/1687952347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 0x7f0f38111580 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 --2- 172.21.15.86:0/1687952347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f38072ad0 0x7f0f38107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 --2- 172.21.15.86:0/1687952347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 -- 172.21.15.86:0/1687952347 >> 172.21.15.86:0/1687952347 conn(0x7f0f3806dd70 msgr2=0x7f0f38070190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 -- 172.21.15.86:0/1687952347 shutdown_connections 2024-03-22T19:18:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.315+0000 7f0f3d200700 1 -- 172.21.15.86:0/1687952347 wait complete. 2024-03-22T19:18:48.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.316+0000 7f0f3d200700 1 Processor -- start 2024-03-22T19:18:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.317+0000 7f0f3d200700 1 -- start start 2024-03-22T19:18:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.318+0000 7f0f3d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f38072ad0 0x7f0f3810a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.318+0000 7f0f3d200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.319+0000 7f0f3d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 0x7f0f381095c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f36a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f36a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49260/0 (socket says 172.21.15.86:49260) 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f36a00700 1 -- 172.21.15.86:0/3063005357 learned_addr learned my addr 172.21.15.86:0/3063005357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38109d60 con 0x7f0f3810f0f0 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38109ed0 con 0x7f0f38072ad0 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f381a9330 con 0x7f0f381082f0 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.320+0000 7f0f37e00700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 0x7f0f381095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.321+0000 7f0f36a00700 1 -- 172.21.15.86:0/3063005357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f38072ad0 msgr2=0x7f0f3810a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.321+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f38072ad0 0x7f0f3810a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.321+0000 7f0f36a00700 1 -- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 msgr2=0x7f0f381095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.321+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 0x7f0f381095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.321+0000 7f0f36a00700 1 -- 172.21.15.86:0/3063005357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30009c40 con 0x7f0f381082f0 2024-03-22T19:18:48.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.322+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38109080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d950 tx=0x7f0f2400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.322+0000 7f0f2fe00700 1 -- 172.21.15.86:0/3063005357 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24009930 con 0x7f0f381082f0 2024-03-22T19:18:48.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.322+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f381a9560 con 0x7f0f381082f0 2024-03-22T19:18:48.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.322+0000 7f0f2fe00700 1 -- 172.21.15.86:0/3063005357 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f24010460 con 0x7f0f381082f0 2024-03-22T19:18:48.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.322+0000 7f0f2fe00700 1 -- 172.21.15.86:0/3063005357 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400f620 con 0x7f0f381082f0 2024-03-22T19:18:48.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.322+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f381a9ac0 con 0x7f0f381082f0 2024-03-22T19:18:48.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.331+0000 7f0f2fe00700 1 -- 172.21.15.86:0/3063005357 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0f24009a90 con 0x7f0f381082f0 2024-03-22T19:18:48.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.334+0000 7f0f2fe00700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f1c0a0b10 0x7f0f1c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.334+0000 7f0f2fe00700 1 -- 172.21.15.86:0/3063005357 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f0f240cbfc0 con 0x7f0f381082f0 2024-03-22T19:18:48.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.336+0000 7f0f52376b80 1 Processor -- start 2024-03-22T19:18:48.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.336+0000 7f0f37400700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f1c0a0b10 0x7f0f1c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.336+0000 7f0f52376b80 1 -- start start 2024-03-22T19:18:48.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.336+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x558670691a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.336+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x558670699a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f52376b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586705e1990 con 0x5586706931f0 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f52376b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586705df350 con 0x558670692830 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f52376b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586705dac40 con 0x5586706915f0 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:33898/0 (socket says 172.21.15.86:33898) 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 -- 172.21.15.86:0/274474167 learned_addr learned my addr 172.21.15.86:0/274474167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 -- 172.21.15.86:0/274474167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 msgr2=0x558670691a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x558670691a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 -- 172.21.15.86:0/274474167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 msgr2=0x558670699a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x558670699a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f36a00700 1 -- 172.21.15.86:0/274474167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55867045c3b0 con 0x558670692830 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.337+0000 7f0f37400700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f1c0a0b10 0x7f0f1c0a2fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0f38069dc0 tx=0x7f0f28009e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.338+0000 7f0f36a00700 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692cb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f240a1630 tx=0x7f0f24099220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6316bd14c755d34f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.349+0000 7f0f2cc00700 1 -- 172.21.15.86:0/274474167 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240996e0 con 0x558670692830 2024-03-22T19:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.349+0000 7f0f2cc00700 1 -- 172.21.15.86:0/274474167 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2409d510 con 0x558670692830 2024-03-22T19:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.349+0000 7f0f2cc00700 1 -- 172.21.15.86:0/274474167 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240b90b0 con 0x558670692830 2024-03-22T19:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.350+0000 7f0f52376b80 1 -- 172.21.15.86:0/274474167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 msgr2=0x558670692cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.350+0000 7f0f52376b80 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692cb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f240a1630 tx=0x7f0f24099220 comp rx=0 tx=0).stop 2024-03-22T19:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.351+0000 7f0f52376b80 1 -- 172.21.15.86:0/274474167 shutdown_connections 2024-03-22T19:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.351+0000 7f0f52376b80 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x558670699a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.351+0000 7f0f52376b80 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x558670691a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.351+0000 7f0f52376b80 1 --2- 172.21.15.86:0/274474167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692cb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.351+0000 7f0f52376b80 1 -- 172.21.15.86:0/274474167 >> 172.21.15.86:0/274474167 conn(0x5586705fe220 msgr2=0x558670683e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.352+0000 7f0f52376b80 1 -- 172.21.15.86:0/274474167 shutdown_connections 2024-03-22T19:18:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.353+0000 7f0f52376b80 1 -- 172.21.15.86:0/274474167 wait complete. 2024-03-22T19:18:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.353+0000 7f0f52376b80 1 -- start start 2024-03-22T19:18:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.354+0000 7f0f52376b80 1 Processor -- start 2024-03-22T19:18:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.354+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x55867069b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.354+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x55867069bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x5586706a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f52376b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558670576d80 con 0x5586706931f0 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f52376b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558670576a20 con 0x558670692830 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f52376b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867045e6a0 con 0x5586706915f0 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f52376b80 10 client.0 _ll_register_callbacks cb 0x5586703e23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f37400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x55867069b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f37400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x55867069b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:49284/0 (socket says 172.21.15.86:49284) 2024-03-22T19:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.355+0000 7f0f37400700 1 -- 172.21.15.86:0/1266586842 learned_addr learned my addr 172.21.15.86:0/1266586842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f36a00700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x55867069bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f37e00700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x5586706a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f36a00700 1 -- 172.21.15.86:0/1266586842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 msgr2=0x55867069b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f36a00700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x55867069b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f36a00700 1 -- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 msgr2=0x5586706a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f36a00700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x5586706a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f36a00700 1 -- 172.21.15.86:0/1266586842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f381a9ac0 con 0x558670692830 2024-03-22T19:18:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.356+0000 7f0f37e00700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x5586706a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:18:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.358+0000 7f0f36a00700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x55867069bb40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f24099670 tx=0x7f0f24095b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5925582ae0eadf78 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.358+0000 7f0f21600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:48.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.370+0000 7f0f21600700 1 -- 172.21.15.86:0/1266586842 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240996e0 con 0x558670692830 2024-03-22T19:18:48.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.370+0000 7f0f21600700 1 -- 172.21.15.86:0/1266586842 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f0f240b90b0 con 0x558670692830 2024-03-22T19:18:48.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.370+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55867067bc30 con 0x558670692830 2024-03-22T19:18:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.374+0000 7f0f21600700 1 -- 172.21.15.86:0/1266586842 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f24010820 con 0x558670692830 2024-03-22T19:18:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.374+0000 7f0f52376b80 10 client.26360 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:48.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.374+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558670408b60 con 0x558670692830 2024-03-22T19:18:48.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.379+0000 7f0f21600700 1 -- 172.21.15.86:0/1266586842 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2407e080 con 0x558670692830 2024-03-22T19:18:48.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.380+0000 7f0f21600700 1 -- 172.21.15.86:0/1266586842 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0f24098bd0 con 0x558670692830 2024-03-22T19:18:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.382+0000 7f0f52376b80 10 client.26360 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.382+0000 7f0f52376b80 10 client.26360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:18:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.382+0000 7f0f52376b80 20 client.26360 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.382+0000 7f0f52376b80 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5586706a6f50 0x5586706a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.382+0000 7f0f52376b80 4 client.26360 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.382+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586705d3700 con 0x5586706a6f50 2024-03-22T19:18:48.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.383+0000 7f0f37400700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5586706a6f50 0x5586706a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.385+0000 7f0f21600700 10 client.26360 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:18:48.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.385+0000 7f0f37400700 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5586706a6f50 0x5586706a9370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7a35609aa41cd0b server_cookie=6ebcde962880d331 in_seq=0 out_seq=0 2024-03-22T19:18:48.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.387+0000 7f0f21600700 1 -- 172.21.15.86:0/1266586842 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5586705d3700 con 0x5586706a6f50 2024-03-22T19:18:48.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.387+0000 7f0f21600700 10 client.26360 handle_command_reply: tid=0 2024-03-22T19:18:48.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.387+0000 7f0f52376b80 1 client.26360 shutdown 2024-03-22T19:18:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.387+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 msgr2=0x55867069bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.387+0000 7f0f52376b80 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x55867069bb40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f24099670 tx=0x7f0f24095b50 comp rx=0 tx=0).stop 2024-03-22T19:18:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 shutdown_connections 2024-03-22T19:18:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5586706a6f50 0x5586706a9370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706931f0 0x5586706a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706915f0 0x55867069b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 --2- 172.21.15.86:0/1266586842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x55867069bb40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 >> 172.21.15.86:0/1266586842 conn(0x5586705fe220 msgr2=0x558670683350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f21600700 0 client.26360 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:18:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 shutdown_connections 2024-03-22T19:18:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 1 -- 172.21.15.86:0/1266586842 wait complete. 2024-03-22T19:18:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.388+0000 7f0f52376b80 20 client.26360 trim_cache size 0 max 16384 2024-03-22T19:18:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.422+0000 7f0f52376b80 1 Processor -- start 2024-03-22T19:18:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.422+0000 7f0f52376b80 1 -- start start 2024-03-22T19:18:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 0x5586706a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x55867069a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f52376b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558670691c40 con 0x5586706915f0 2024-03-22T19:18:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f52376b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558670680740 con 0x558670692830 2024-03-22T19:18:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f52376b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867067e3c0 con 0x5586706931f0 2024-03-22T19:18:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.423+0000 7f0f37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 0x5586706a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.425+0000 7f0f37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 0x5586706a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45720/0 (socket says 172.21.15.86:45720) 2024-03-22T19:18:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.425+0000 7f0f37400700 1 -- 172.21.15.86:0/2799819848 learned_addr learned my addr 172.21.15.86:0/2799819848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.425+0000 7f0f37e00700 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x55867069a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.425+0000 7f0f37400700 1 -- 172.21.15.86:0/2799819848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 msgr2=0x55867069a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.425+0000 7f0f37400700 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x55867069a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.425+0000 7f0f37400700 1 -- 172.21.15.86:0/2799819848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 msgr2=0x558670692c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:18:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.426+0000 7f0f37400700 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.426+0000 7f0f37400700 1 -- 172.21.15.86:0/2799819848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586705a0ea0 con 0x5586706915f0 2024-03-22T19:18:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.426+0000 7f0f37400700 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 0x5586706a0a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800d6c0 tx=0x7f0f28009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7a6cd757188a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.427+0000 7f0f2d600700 1 -- 172.21.15.86:0/2799819848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28012b10 con 0x5586706915f0 2024-03-22T19:18:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.427+0000 7f0f2d600700 1 -- 172.21.15.86:0/2799819848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f28012c70 con 0x5586706915f0 2024-03-22T19:18:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f2d600700 1 -- 172.21.15.86:0/2799819848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28010670 con 0x5586706915f0 2024-03-22T19:18:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 -- 172.21.15.86:0/2799819848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 msgr2=0x5586706a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 0x5586706a0a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800d6c0 tx=0x7f0f28009f90 comp rx=0 tx=0).stop 2024-03-22T19:18:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 -- 172.21.15.86:0/2799819848 shutdown_connections 2024-03-22T19:18:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x55867069a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586706915f0 0x5586706a0a90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 --2- 172.21.15.86:0/2799819848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558670692830 0x558670692c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.428+0000 7f0f52376b80 1 -- 172.21.15.86:0/2799819848 >> 172.21.15.86:0/2799819848 conn(0x5586705fe220 msgr2=0x55867069ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.429+0000 7f0f52376b80 1 -- 172.21.15.86:0/2799819848 shutdown_connections 2024-03-22T19:18:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.429+0000 7f0f52376b80 1 -- 172.21.15.86:0/2799819848 wait complete. 2024-03-22T19:18:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.431+0000 7f0f52376b80 1 -- start start 2024-03-22T19:18:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.431+0000 7f0f52376b80 1 Processor -- start 2024-03-22T19:18:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 0x5586706f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558670692830 0x5586706f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x5586706f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586706f7b40 con 0x558670692830 2024-03-22T19:18:48.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586706f7cb0 con 0x5586706915f0 2024-03-22T19:18:48.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558670688b00 con 0x5586706931f0 2024-03-22T19:18:48.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f52376b80 10 client.0 _ll_register_callbacks cb 0x7f0f240995a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:48.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.432+0000 7f0f37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 0x5586706f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.433+0000 7f0f37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 0x5586706f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44074/0 (socket says 172.21.15.86:44074) 2024-03-22T19:18:48.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.433+0000 7f0f37400700 1 -- 172.21.15.86:0/3887546560 learned_addr learned my addr 172.21.15.86:0/3887546560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.433+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558670692830 0x5586706f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.434+0000 7f0f37400700 1 -- 172.21.15.86:0/3887546560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 msgr2=0x5586706f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.434+0000 7f0f37400700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x5586706f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.434+0000 7f0f37400700 1 -- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558670692830 msgr2=0x5586706f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.434+0000 7f0f37400700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558670692830 0x5586706f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.434+0000 7f0f37400700 1 -- 172.21.15.86:0/3887546560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2800d870 con 0x5586706915f0 2024-03-22T19:18:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.435+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558670692830 0x5586706f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:18:48.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.436+0000 7f0f37400700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 0x5586706f2a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f28005c80 tx=0x7f0f28012910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adeaf2117fcbad25 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.436+0000 7f0f21600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:48.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.437+0000 7f0f21600700 1 -- 172.21.15.86:0/3887546560 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28020860 con 0x5586706915f0 2024-03-22T19:18:48.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.437+0000 7f0f21600700 1 -- 172.21.15.86:0/3887546560 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f0f28019040 con 0x5586706915f0 2024-03-22T19:18:48.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.437+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f2409d510 con 0x5586706915f0 2024-03-22T19:18:48.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.438+0000 7f0f21600700 1 -- 172.21.15.86:0/3887546560 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f2802f450 con 0x5586706915f0 2024-03-22T19:18:48.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.438+0000 7f0f52376b80 10 client.26363 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.438+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558670460cd0 con 0x5586706915f0 2024-03-22T19:18:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f21600700 1 -- 172.21.15.86:0/3887546560 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2409d510 con 0x5586706915f0 2024-03-22T19:18:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f21600700 1 -- 172.21.15.86:0/3887546560 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0f28010710 con 0x5586706915f0 2024-03-22T19:18:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f52376b80 10 client.26363 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f52376b80 10 client.26363 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:18:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f52376b80 20 client.26363 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f52376b80 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55867068a430 0x55867068c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f52376b80 4 client.26363 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:18:48.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.439+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558670429cf0 con 0x55867068a430 2024-03-22T19:18:48.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.440+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55867068a430 0x55867068c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.442+0000 7f0f36a00700 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55867068a430 0x55867068c850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b1a68c81942a9fa server_cookie=d737e63b50fc6cf3 in_seq=0 out_seq=0 2024-03-22T19:18:48.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.442+0000 7f0f21600700 10 client.26363 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:18:48.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.443+0000 7f0f21600700 1 -- 172.21.15.86:0/3887546560 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558670429cf0 con 0x55867068a430 2024-03-22T19:18:48.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.443+0000 7f0f21600700 10 client.26363 handle_command_reply: tid=0 2024-03-22T19:18:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.443+0000 7f0f52376b80 1 client.26363 shutdown 2024-03-22T19:18:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.443+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 msgr2=0x5586706f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.443+0000 7f0f52376b80 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 0x5586706f2a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f28005c80 tx=0x7f0f28012910 comp rx=0 tx=0).stop 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 shutdown_connections 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f52376b80 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x55867068a430 0x55867068c850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f52376b80 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586706931f0 0x5586706f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f52376b80 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586706915f0 0x5586706f2a30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f52376b80 1 --2- 172.21.15.86:0/3887546560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558670692830 0x5586706f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 >> 172.21.15.86:0/3887546560 conn(0x5586705fe220 msgr2=0x55867069a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.445+0000 7f0f21600700 0 client.26363 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:18:48.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.446+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 shutdown_connections 2024-03-22T19:18:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.447+0000 7f0f52376b80 1 -- 172.21.15.86:0/3887546560 wait complete. 2024-03-22T19:18:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.447+0000 7f0f52376b80 20 client.26363 trim_cache size 0 max 16384 2024-03-22T19:18:48.448 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:18:48.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.448+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f1c0a0b10 msgr2=0x7f0f1c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.448+0000 7f0f3d200700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f1c0a0b10 0x7f0f1c0a2fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0f38069dc0 tx=0x7f0f28009e70 comp rx=0 tx=0).stop 2024-03-22T19:18:48.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.448+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 msgr2=0x7f0f38109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.448+0000 7f0f3d200700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38109080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d950 tx=0x7f0f2400dc60 comp rx=0 tx=0).stop 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 shutdown_connections 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0f1c0a0b10 0x7f0f1c0a2fd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3810f0f0 0x7f0f381095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f38072ad0 0x7f0f3810a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 --2- 172.21.15.86:0/3063005357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f381082f0 0x7f0f38109080 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 >> 172.21.15.86:0/3063005357 conn(0x7f0f3806dd70 msgr2=0x7f0f3810b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 shutdown_connections 2024-03-22T19:18:48.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.449+0000 7f0f3d200700 1 -- 172.21.15.86:0/3063005357 wait complete. 2024-03-22T19:18:48.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:18:48.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.891+0000 7f6e52200700 1 -- 172.21.15.86:0/3133452045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c108a60 msgr2=0x7f6e4c071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.891+0000 7f6e52200700 1 --2- 172.21.15.86:0/3133452045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c108a60 0x7f6e4c071ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0098f0 tx=0x7f6e3c009c00 comp rx=0 tx=0).stop 2024-03-22T19:18:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.893+0000 7f6e52200700 1 -- 172.21.15.86:0/3133452045 shutdown_connections 2024-03-22T19:18:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.893+0000 7f6e52200700 1 --2- 172.21.15.86:0/3133452045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4c10f1d0 0x7f6e4c111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.893+0000 7f6e52200700 1 --2- 172.21.15.86:0/3133452045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c108a60 0x7f6e4c071ed0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.893+0000 7f6e52200700 1 --2- 172.21.15.86:0/3133452045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.893+0000 7f6e52200700 1 -- 172.21.15.86:0/3133452045 >> 172.21.15.86:0/3133452045 conn(0x7f6e4c06dd20 msgr2=0x7f6e4c070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.893+0000 7f6e52200700 1 -- 172.21.15.86:0/3133452045 shutdown_connections 2024-03-22T19:18:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.894+0000 7f6e52200700 1 -- 172.21.15.86:0/3133452045 wait complete. 2024-03-22T19:18:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.897+0000 7f6e52200700 1 Processor -- start 2024-03-22T19:18:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.897+0000 7f6e52200700 1 -- start start 2024-03-22T19:18:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e52200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c10a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e52200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4c108a60 0x7f6e4c109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e52200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4c10f1d0 0x7f6e4c1095c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e52200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c109d60 con 0x7f6e4c0724a0 2024-03-22T19:18:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e52200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c109ed0 con 0x7f6e4c10f1d0 2024-03-22T19:18:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e52200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1a9620 con 0x7f6e4c108a60 2024-03-22T19:18:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c10a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c10a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45748/0 (socket says 172.21.15.86:45748) 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e4b400700 1 -- 172.21.15.86:0/48295770 learned_addr learned my addr 172.21.15.86:0/48295770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.898+0000 7f6e4be00700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4c10f1d0 0x7f6e4c1095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.899+0000 7f6e4b400700 1 -- 172.21.15.86:0/48295770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4c108a60 msgr2=0x7f6e4c109080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.899+0000 7f6e4b400700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4c108a60 0x7f6e4c109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.899+0000 7f6e4b400700 1 -- 172.21.15.86:0/48295770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4c10f1d0 msgr2=0x7f6e4c1095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.899+0000 7f6e4b400700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4c10f1d0 0x7f6e4c1095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.899+0000 7f6e4b400700 1 -- 172.21.15.86:0/48295770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c009520 con 0x7f6e4c0724a0 2024-03-22T19:18:48.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.900+0000 7f6e4b400700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c10a9f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c000c00 tx=0x7f6e3c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.902+0000 7f6e43e00700 1 -- 172.21.15.86:0/48295770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c01c070 con 0x7f6e4c0724a0 2024-03-22T19:18:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.902+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c1a9850 con 0x7f6e4c0724a0 2024-03-22T19:18:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.902+0000 7f6e43e00700 1 -- 172.21.15.86:0/48295770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e3c0054f0 con 0x7f6e4c0724a0 2024-03-22T19:18:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.902+0000 7f6e43e00700 1 -- 172.21.15.86:0/48295770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c01c070 con 0x7f6e4c0724a0 2024-03-22T19:18:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.902+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c1a9bc0 con 0x7f6e4c0724a0 2024-03-22T19:18:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.909+0000 7f6e43e00700 1 -- 172.21.15.86:0/48295770 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6e3c004cc0 con 0x7f6e4c0724a0 2024-03-22T19:18:48.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.913+0000 7f6e65cdcb80 1 Processor -- start 2024-03-22T19:18:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.913+0000 7f6e65cdcb80 1 -- start start 2024-03-22T19:18:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.913+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1a9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.914+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.914+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1aac00 0x5617cf1ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.914+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf0f9990 con 0x5617cf1aac00 2024-03-22T19:18:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.914+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf0f7350 con 0x5617cf1a9370 2024-03-22T19:18:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.914+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf0f2c40 con 0x5617cf1a9cd0 2024-03-22T19:18:48.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.915+0000 7f6e43e00700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e300a0ba0 0x7f6e300a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.915+0000 7f6e43e00700 1 -- 172.21.15.86:0/48295770 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6e3c0ccf30 con 0x7f6e4c0724a0 2024-03-22T19:18:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43740/0 (socket says 172.21.15.86:43740) 2024-03-22T19:18:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 -- 172.21.15.86:0/767954298 learned_addr learned my addr 172.21.15.86:0/767954298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1aac00 0x5617cf1ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 -- 172.21.15.86:0/767954298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 msgr2=0x5617cf1a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 -- 172.21.15.86:0/767954298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1aac00 msgr2=0x5617cf1ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1aac00 0x5617cf1ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4b400700 1 -- 172.21.15.86:0/767954298 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617cef743b0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.916+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1aac00 0x5617cf1ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T19:18:48.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.917+0000 7f6e4be00700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e300a0ba0 0x7f6e300a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.917+0000 7f6e4b400700 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1aa670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c096b40 tx=0x7f6e3c09a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e37005f2117bd39 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.917+0000 7f6e37e00700 1 -- 172.21.15.86:0/767954298 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c09a640 con 0x5617cf1a9cd0 2024-03-22T19:18:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.917+0000 7f6e37e00700 1 -- 172.21.15.86:0/767954298 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e3c09e470 con 0x5617cf1a9cd0 2024-03-22T19:18:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.917+0000 7f6e37e00700 1 -- 172.21.15.86:0/767954298 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c09e120 con 0x5617cf1a9cd0 2024-03-22T19:18:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.919+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/767954298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 msgr2=0x5617cf1aa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.919+0000 7f6e4be00700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e300a0ba0 0x7f6e300a3060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6e440095b0 tx=0x7f6e44009050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.919+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1aa670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c096b40 tx=0x7f6e3c09a200 comp rx=0 tx=0).stop 2024-03-22T19:18:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/767954298 shutdown_connections 2024-03-22T19:18:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1aac00 0x5617cf1ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/767954298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1aa670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/767954298 >> 172.21.15.86:0/767954298 conn(0x5617cf116220 msgr2=0x5617cf19b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/767954298 shutdown_connections 2024-03-22T19:18:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.921+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/767954298 wait complete. 2024-03-22T19:18:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.923+0000 7f6e65cdcb80 1 -- start start 2024-03-22T19:18:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.923+0000 7f6e65cdcb80 1 Processor -- start 2024-03-22T19:18:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1aac00 0x5617cf1ba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf08ed80 con 0x5617cf1a9cd0 2024-03-22T19:18:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf08ea20 con 0x5617cf1a9370 2024-03-22T19:18:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cef766a0 con 0x5617cf1aac00 2024-03-22T19:18:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.926+0000 7f6e65cdcb80 10 client.0 _ll_register_callbacks cb 0x5617ceefa3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.927+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.927+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44104/0 (socket says 172.21.15.86:44104) 2024-03-22T19:18:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.927+0000 7f6e4b400700 1 -- 172.21.15.86:0/4280630598 learned_addr learned my addr 172.21.15.86:0/4280630598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.927+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.928+0000 7f6e4aa00700 1 -- 172.21.15.86:0/4280630598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1aac00 msgr2=0x5617cf1ba520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T19:18:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.928+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1aac00 0x5617cf1ba520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.928+0000 7f6e4aa00700 1 -- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 msgr2=0x5617cf1b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.928+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.928+0000 7f6e4aa00700 1 -- 172.21.15.86:0/4280630598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c1a9bc0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.929+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b35d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c150 tx=0x7f6e3800c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc62fee55d109164 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.929+0000 7f6e34c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.929+0000 7f6e34c00700 1 -- 172.21.15.86:0/4280630598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380113f0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.930+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617cf193ad0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.930+0000 7f6e34c00700 1 -- 172.21.15.86:0/4280630598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6e38011550 con 0x5617cf1a9cd0 2024-03-22T19:18:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.934+0000 7f6e34c00700 1 -- 172.21.15.86:0/4280630598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e38011750 con 0x5617cf1a9cd0 2024-03-22T19:18:48.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.934+0000 7f6e65cdcb80 10 client.16245 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:48.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.934+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5617cef20b60 con 0x5617cf1a9cd0 2024-03-22T19:18:48.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.934+0000 7f6e34c00700 1 -- 172.21.15.86:0/4280630598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617cf193ad0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e34c00700 1 -- 172.21.15.86:0/4280630598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e38019a60 con 0x5617cf1a9cd0 2024-03-22T19:18:48.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e65cdcb80 10 client.16245 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e65cdcb80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e65cdcb80 20 client.16245 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf1bee20 0x5617cf1c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e65cdcb80 4 client.16245 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.935+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617cf0eb700 con 0x5617cf1bee20 2024-03-22T19:18:48.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.939+0000 7f6e4b400700 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf1bee20 0x5617cf1c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.948+0000 7f6e4b400700 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf1bee20 0x5617cf1c1240 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15248bc15014c1c6 server_cookie=5b21f5433853d417 in_seq=0 out_seq=0 2024-03-22T19:18:48.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.948+0000 7f6e34c00700 10 client.16245 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:48.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.953+0000 7f6e34c00700 1 -- 172.21.15.86:0/4280630598 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5617cf0eb700 con 0x5617cf1bee20 2024-03-22T19:18:48.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.953+0000 7f6e34c00700 10 client.16245 handle_command_reply: tid=0 2024-03-22T19:18:48.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.953+0000 7f6e65cdcb80 1 client.16245 shutdown 2024-03-22T19:18:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9cd0 msgr2=0x5617cf1b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b35d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c150 tx=0x7f6e3800c460 comp rx=0 tx=0).stop 2024-03-22T19:18:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 shutdown_connections 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf1bee20 0x5617cf1c1240 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1aac00 0x5617cf1ba520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1a9370 0x5617cf1b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/4280630598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b35d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 >> 172.21.15.86:0/4280630598 conn(0x5617cf116220 msgr2=0x5617cf1a8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.955+0000 7f6e34c00700 0 client.16245 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.956+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 shutdown_connections 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.956+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/4280630598 wait complete. 2024-03-22T19:18:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.956+0000 7f6e65cdcb80 20 client.16245 trim_cache size 0 max 16384 2024-03-22T19:18:48.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.987+0000 7f6e65cdcb80 1 Processor -- start 2024-03-22T19:18:48.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.987+0000 7f6e65cdcb80 1 -- start start 2024-03-22T19:18:48.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf1aa110 con 0x5617cf1a9370 2024-03-22T19:18:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf19c910 con 0x5617cf1aac00 2024-03-22T19:18:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf19cd40 con 0x5617cf1a9cd0 2024-03-22T19:18:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e4aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e4be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.990+0000 7f6e4be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1babd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44106/0 (socket says 172.21.15.86:44106) 2024-03-22T19:18:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.989+0000 7f6e4aa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1ba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43746/0 (socket says 172.21.15.86:43746) 2024-03-22T19:18:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.990+0000 7f6e4aa00700 1 -- 172.21.15.86:0/2431904744 learned_addr learned my addr 172.21.15.86:0/2431904744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.991+0000 7f6e4aa00700 1 -- 172.21.15.86:0/2431904744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 msgr2=0x5617cf1babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.991+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.991+0000 7f6e4aa00700 1 -- 172.21.15.86:0/2431904744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 msgr2=0x5617cf1b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.991+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.991+0000 7f6e4aa00700 1 -- 172.21.15.86:0/2431904744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617cf0b8ea0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.993+0000 7f6e4aa00700 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1ba4e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e38001eb0 tx=0x7f6e38006cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899088bdb9c2d7ed server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.995+0000 7f6e41200700 1 -- 172.21.15.86:0/2431904744 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380106e0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.995+0000 7f6e41200700 1 -- 172.21.15.86:0/2431904744 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e38010d00 con 0x5617cf1a9cd0 2024-03-22T19:18:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.995+0000 7f6e41200700 1 -- 172.21.15.86:0/2431904744 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38019ad0 con 0x5617cf1a9cd0 2024-03-22T19:18:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.996+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/2431904744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 msgr2=0x5617cf1ba4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.996+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1ba4e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e38001eb0 tx=0x7f6e38006cf0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.997+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/2431904744 shutdown_connections 2024-03-22T19:18:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.997+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.997+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.997+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/2431904744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1ba4e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.997+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/2431904744 >> 172.21.15.86:0/2431904744 conn(0x5617cf116220 msgr2=0x5617cf1b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.998+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/2431904744 shutdown_connections 2024-03-22T19:18:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.998+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/2431904744 wait complete. 2024-03-22T19:18:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.998+0000 7f6e65cdcb80 1 -- start start 2024-03-22T19:18:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:48.998+0000 7f6e65cdcb80 1 Processor -- start 2024-03-22T19:18:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.000+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.000+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.000+0000 7f6e65cdcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf1b9df0 con 0x5617cf1a9370 2024-03-22T19:18:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf1b9f60 con 0x5617cf1aac00 2024-03-22T19:18:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e65cdcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617cf1ba120 con 0x5617cf1a9cd0 2024-03-22T19:18:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e65cdcb80 10 client.0 _ll_register_callbacks cb 0x5617ceefa3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e4b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45796/0 (socket says 172.21.15.86:45796) 2024-03-22T19:18:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.001+0000 7f6e4b400700 1 -- 172.21.15.86:0/175442072 learned_addr learned my addr 172.21.15.86:0/175442072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.002+0000 7f6e4be00700 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.002+0000 7f6e4be00700 1 -- 172.21.15.86:0/175442072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 msgr2=0x5617cf1b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.002+0000 7f6e4be00700 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.002+0000 7f6e4be00700 1 -- 172.21.15.86:0/175442072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 msgr2=0x5617cf1b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.002+0000 7f6e4be00700 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.003+0000 7f6e4be00700 1 -- 172.21.15.86:0/175442072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617cef20b60 con 0x5617cf1aac00 2024-03-22T19:18:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.005+0000 7f6e4be00700 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1b98b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5617cf1b2bb0 tx=0x7f6e4400b070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5532898bfcffc475 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.005+0000 7f6e34c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.006+0000 7f6e34c00700 1 -- 172.21.15.86:0/175442072 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015e30 con 0x5617cf1aac00 2024-03-22T19:18:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.006+0000 7f6e34c00700 1 -- 172.21.15.86:0/175442072 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6e440149c0 con 0x5617cf1aac00 2024-03-22T19:18:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.006+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e3c09e470 con 0x5617cf1aac00 2024-03-22T19:18:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.007+0000 7f6e34c00700 1 -- 172.21.15.86:0/175442072 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e44014b80 con 0x5617cf1aac00 2024-03-22T19:18:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.007+0000 7f6e65cdcb80 10 client.26372 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.007+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5617cef78cd0 con 0x5617cf1aac00 2024-03-22T19:18:49.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.008+0000 7f6e34c00700 1 -- 172.21.15.86:0/175442072 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e3c09e470 con 0x5617cf1aac00 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e34c00700 1 -- 172.21.15.86:0/175442072 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e44021420 con 0x5617cf1aac00 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e65cdcb80 10 client.26372 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e65cdcb80 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e65cdcb80 20 client.26372 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf211a90 0x5617cf213eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e65cdcb80 4 client.26372 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.011+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5617cef41cf0 con 0x5617cf211a90 2024-03-22T19:18:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.012+0000 7f6e4b400700 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf211a90 0x5617cf213eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.015+0000 7f6e34c00700 10 client.26372 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.015+0000 7f6e4b400700 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf211a90 0x5617cf213eb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1888f0b32aebd79 server_cookie=fa1e577353893495 in_seq=0 out_seq=0 2024-03-22T19:18:49.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.016+0000 7f6e34c00700 1 -- 172.21.15.86:0/175442072 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5617cef41cf0 con 0x5617cf211a90 2024-03-22T19:18:49.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.016+0000 7f6e34c00700 10 client.26372 handle_command_reply: tid=0 2024-03-22T19:18:49.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.016+0000 7f6e65cdcb80 1 client.26372 shutdown 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 msgr2=0x5617cf1b98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1b98b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5617cf1b2bb0 tx=0x7f6e4400b070 comp rx=0 tx=0).stop 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 shutdown_connections 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5617cf211a90 0x5617cf213eb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617cf1aac00 0x5617cf1b98b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617cf1a9370 0x5617cf1b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 --2- 172.21.15.86:0/175442072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617cf1a9cd0 0x5617cf1b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 >> 172.21.15.86:0/175442072 conn(0x5617cf116220 msgr2=0x5617cf1c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e34c00700 0 client.26372 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 shutdown_connections 2024-03-22T19:18:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.017+0000 7f6e65cdcb80 1 -- 172.21.15.86:0/175442072 wait complete. 2024-03-22T19:18:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.018+0000 7f6e65cdcb80 20 client.26372 trim_cache size 0 max 16384 2024-03-22T19:18:49.019 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:18:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e300a0ba0 msgr2=0x7f6e300a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e300a0ba0 0x7f6e300a3060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6e440095b0 tx=0x7f6e44009050 comp rx=0 tx=0).stop 2024-03-22T19:18:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 msgr2=0x7f6e4c10a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c10a9f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c000c00 tx=0x7f6e3c00b8f0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 shutdown_connections 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6e300a0ba0 0x7f6e300a3060 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4c10f1d0 0x7f6e4c1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0724a0 0x7f6e4c10a9f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 --2- 172.21.15.86:0/48295770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4c108a60 0x7f6e4c109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 >> 172.21.15.86:0/48295770 conn(0x7f6e4c06dd20 msgr2=0x7f6e4c10b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.019+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 shutdown_connections 2024-03-22T19:18:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:49.020+0000 7f6e52200700 1 -- 172.21.15.86:0/48295770 wait complete. 2024-03-22T19:18:49.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:49 smithi167 ceph-mon[47728]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 90 MiB/s wr, 14.44k op/s 2024-03-22T19:18:49.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:49 smithi131 ceph-mon[78559]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 90 MiB/s wr, 14.44k op/s 2024-03-22T19:18:50.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:49 smithi086 ceph-mon[38032]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 90 MiB/s wr, 14.44k op/s 2024-03-22T19:18:50.059 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-22T19:18:50.060 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-22T19:18:51.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:51 smithi167 ceph-mon[47728]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 82 MiB/s wr, 13.69k op/s 2024-03-22T19:18:51.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:51 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:18:51.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:51 smithi131 ceph-mon[78559]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 82 MiB/s wr, 13.69k op/s 2024-03-22T19:18:51.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:51 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:18:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:51 smithi086 ceph-mon[38032]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 82 MiB/s wr, 13.69k op/s 2024-03-22T19:18:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:51 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:18:53.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 79 MiB/s wr, 13.62k op/s 2024-03-22T19:18:53.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:53.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:53 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 79 MiB/s wr, 13.62k op/s 2024-03-22T19:18:54.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:53 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 79 MiB/s wr, 13.62k op/s 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.284 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.285 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.285 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:53 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi131 to 182.4M 2024-03-22T19:18:54.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: Unable to set osd_memory_target on smithi131 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-03-22T19:18:54.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:54.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:54 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi131 to 182.4M 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: Unable to set osd_memory_target on smithi131 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:54 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi131 to 182.4M 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: Unable to set osd_memory_target on smithi131 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.016 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.017 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:55.017 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:54 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:56.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:55 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi167 to 182.5M 2024-03-22T19:18:56.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:55 smithi086 ceph-mon[38032]: Unable to set osd_memory_target on smithi167 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-22T19:18:56.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:55 smithi086 ceph-mon[38032]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 MiB/s wr, 15.56k op/s 2024-03-22T19:18:56.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:55 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi167 to 182.5M 2024-03-22T19:18:56.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:55 smithi131 ceph-mon[78559]: Unable to set osd_memory_target on smithi167 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-22T19:18:56.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:55 smithi131 ceph-mon[78559]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 MiB/s wr, 15.56k op/s 2024-03-22T19:18:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:55 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi167 to 182.5M 2024-03-22T19:18:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:55 smithi167 ceph-mon[47728]: Unable to set osd_memory_target on smithi167 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-22T19:18:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:55 smithi167 ceph-mon[47728]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 MiB/s wr, 15.56k op/s 2024-03-22T19:18:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.223+0000 7f959ae00700 1 -- 172.21.15.86:0/170866766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95940721e0 msgr2=0x7f9594072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.223+0000 7f959ae00700 1 --2- 172.21.15.86:0/170866766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95940721e0 0x7f9594072600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b0a0 tx=0x7f958c00b3b0 comp rx=0 tx=0).stop 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.224+0000 7f959ae00700 1 -- 172.21.15.86:0/170866766 shutdown_connections 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.224+0000 7f959ae00700 1 --2- 172.21.15.86:0/170866766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9594108ee0 0x7f959410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.224+0000 7f959ae00700 1 --2- 172.21.15.86:0/170866766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95940721e0 0x7f9594072600 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.224+0000 7f959ae00700 1 --2- 172.21.15.86:0/170866766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594108520 0x7f95941089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.224+0000 7f959ae00700 1 -- 172.21.15.86:0/170866766 >> 172.21.15.86:0/170866766 conn(0x7f959406dd20 msgr2=0x7f9594070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.224+0000 7f959ae00700 1 -- 172.21.15.86:0/170866766 shutdown_connections 2024-03-22T19:18:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.225+0000 7f959ae00700 1 -- 172.21.15.86:0/170866766 wait complete. 2024-03-22T19:18:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.226+0000 7f959ae00700 1 Processor -- start 2024-03-22T19:18:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.226+0000 7f959ae00700 1 -- start start 2024-03-22T19:18:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f959ae00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95940721e0 0x7f95941a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f959ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 0x7f95941a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f959ae00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9594108ee0 0x7f95941ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f959ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941ad810 con 0x7f9594108520 2024-03-22T19:18:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f959ae00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941ad980 con 0x7f9594108ee0 2024-03-22T19:18:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f959ae00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941adb40 con 0x7f95940721e0 2024-03-22T19:18:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f9598e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9594108ee0 0x7f95941ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.227+0000 7f9593400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 0x7f95941a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.228+0000 7f9593400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 0x7f95941a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45828/0 (socket says 172.21.15.86:45828) 2024-03-22T19:18:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.228+0000 7f9593400700 1 -- 172.21.15.86:0/4242150909 learned_addr learned my addr 172.21.15.86:0/4242150909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.229+0000 7f9593400700 1 -- 172.21.15.86:0/4242150909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95940721e0 msgr2=0x7f95941a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.229+0000 7f9593400700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95940721e0 0x7f95941a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.229+0000 7f9593400700 1 -- 172.21.15.86:0/4242150909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9594108ee0 msgr2=0x7f95941ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.229+0000 7f9593400700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9594108ee0 0x7f95941ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.229+0000 7f9593400700 1 -- 172.21.15.86:0/4242150909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c009d00 con 0x7f9594108520 2024-03-22T19:18:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.230+0000 7f9593400700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 0x7f95941a6230 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f958400b530 tx=0x7f958400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:57.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.232+0000 7f9590c00700 1 -- 172.21.15.86:0/4242150909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584011830 con 0x7f9594108520 2024-03-22T19:18:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.232+0000 7f9590c00700 1 -- 172.21.15.86:0/4242150909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9584011e50 con 0x7f9594108520 2024-03-22T19:18:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.232+0000 7f9590c00700 1 -- 172.21.15.86:0/4242150909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400f2e0 con 0x7f9594108520 2024-03-22T19:18:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.232+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95941ade20 con 0x7f9594108520 2024-03-22T19:18:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.232+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95941ae250 con 0x7f9594108520 2024-03-22T19:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.243+0000 7f9590c00700 1 -- 172.21.15.86:0/4242150909 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9584010580 con 0x7f9594108520 2024-03-22T19:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.245+0000 7f9590c00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95780a0be0 0x7f95780a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:57.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.245+0000 7f9590c00700 1 -- 172.21.15.86:0/4242150909 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f95840cbae0 con 0x7f9594108520 2024-03-22T19:18:57.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.246+0000 7f9593e00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95780a0be0 0x7f95780a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:57.247+0000 7f9593e00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95780a0be0 0x7f95780a30a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b070 tx=0x7f958c007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:57.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 70 MiB/s wr, 12.93k op/s 2024-03-22T19:18:57.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:18:57.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:57 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:18:57.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 70 MiB/s wr, 12.93k op/s 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-22T19:18:57.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:18:57.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:18:57.762 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:57 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:18:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 70 MiB/s wr, 12.93k op/s 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:18:57.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:18:57.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:57 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:18:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.072+0000 7f95ae9a5b80 1 Processor -- start 2024-03-22T19:18:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.073+0000 7f95ae9a5b80 1 -- start start 2024-03-22T19:18:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.075+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 0x5628a720b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.075+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720b600 0x5628a720bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.075+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720c4a0 0x5628a720c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f9593e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 0x5628a720b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f9593e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 0x5628a720b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43776/0 (socket says 172.21.15.86:43776) 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f9593e00700 1 -- 172.21.15.86:0/1661063190 learned_addr learned my addr 172.21.15.86:0/1661063190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a7163a10 con 0x5628a720c4a0 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a71613d0 con 0x5628a720b600 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a715ccc0 con 0x5628a720aca0 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f9593400700 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720b600 0x5628a720bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.076+0000 7f9598e00700 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720c4a0 0x5628a720c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.078+0000 7f9593e00700 1 -- 172.21.15.86:0/1661063190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720b600 msgr2=0x5628a720bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.078+0000 7f9593e00700 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720b600 0x5628a720bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.078+0000 7f9593e00700 1 -- 172.21.15.86:0/1661063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720c4a0 msgr2=0x5628a720c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.078+0000 7f9593e00700 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720c4a0 0x5628a720c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.078+0000 7f9593e00700 1 -- 172.21.15.86:0/1661063190 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628a6fde430 con 0x5628a720aca0 2024-03-22T19:18:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.079+0000 7f9593e00700 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 0x5628a720b0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f958c015000 tx=0x7f958c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6347490e2d25af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.079+0000 7f9589600700 1 -- 172.21.15.86:0/1661063190 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00e040 con 0x5628a720aca0 2024-03-22T19:18:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.079+0000 7f9589600700 1 -- 172.21.15.86:0/1661063190 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f958c009480 con 0x5628a720aca0 2024-03-22T19:18:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.079+0000 7f9589600700 1 -- 172.21.15.86:0/1661063190 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c01bd80 con 0x5628a720aca0 2024-03-22T19:18:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.080+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 msgr2=0x5628a720b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.080+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 0x5628a720b0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f958c015000 tx=0x7f958c019000 comp rx=0 tx=0).stop 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.081+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 shutdown_connections 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.081+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720c4a0 0x5628a720c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.081+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720aca0 0x5628a720b0c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.081+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1661063190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720b600 0x5628a720bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.081+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 >> 172.21.15.86:0/1661063190 conn(0x5628a71802a0 msgr2=0x5628a7205c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.082+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 shutdown_connections 2024-03-22T19:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.082+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1661063190 wait complete. 2024-03-22T19:18:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.082+0000 7f95ae9a5b80 1 -- start start 2024-03-22T19:18:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.083+0000 7f95ae9a5b80 1 Processor -- start 2024-03-22T19:18:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.084+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a7214b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.084+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 0x5628a72150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.084+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.085+0000 7f9593400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 0x5628a72150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.085+0000 7f9593400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 0x5628a72150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43786/0 (socket says 172.21.15.86:43786) 2024-03-22T19:18:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.086+0000 7f9598e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.086+0000 7f9598e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44160/0 (socket says 172.21.15.86:44160) 2024-03-22T19:18:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.087+0000 7f95ae9a5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a70f8e00 con 0x5628a720aca0 2024-03-22T19:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.087+0000 7f95ae9a5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a70f8aa0 con 0x5628a720c4a0 2024-03-22T19:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.087+0000 7f95ae9a5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a6fe0720 con 0x5628a720b600 2024-03-22T19:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.087+0000 7f9593400700 1 -- 172.21.15.86:0/1287295608 learned_addr learned my addr 172.21.15.86:0/1287295608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.087+0000 7f95ae9a5b80 10 client.0 _ll_register_callbacks cb 0x5628a6f64420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f9593400700 1 -- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 msgr2=0x5628a721c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f9593400700 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f9593400700 1 -- 172.21.15.86:0/1287295608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 msgr2=0x5628a7214b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f9593400700 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a7214b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f9593400700 1 -- 172.21.15.86:0/1287295608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959410e1b0 con 0x5628a720b600 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f9593400700 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 0x5628a72150c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9584095610 tx=0x7f95840841a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ceb277091648a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f957e000700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.088+0000 7f957e000700 1 -- 172.21.15.86:0/1287295608 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840b0070 con 0x5628a720b600 2024-03-22T19:18:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.089+0000 7f957e000700 1 -- 172.21.15.86:0/1287295608 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9584099160 con 0x5628a720b600 2024-03-22T19:18:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.089+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628a71fdb50 con 0x5628a720b600 2024-03-22T19:18:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.089+0000 7f957e000700 1 -- 172.21.15.86:0/1287295608 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f958409cf90 con 0x5628a720b600 2024-03-22T19:18:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.089+0000 7f95ae9a5b80 10 client.26365 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.089+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5628a6f8abe0 con 0x5628a720b600 2024-03-22T19:18:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.090+0000 7f957e000700 1 -- 172.21.15.86:0/1287295608 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628a71fdb50 con 0x5628a720b600 2024-03-22T19:18:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.091+0000 7f957e000700 1 -- 172.21.15.86:0/1287295608 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f95840986e0 con 0x5628a720b600 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.091+0000 7f95ae9a5b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.091+0000 7f95ae9a5b80 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.091+0000 7f95ae9a5b80 20 client.26365 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.091+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a7220840 0x5628a7222c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.091+0000 7f95ae9a5b80 4 client.26365 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.092+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628a7155780 con 0x5628a7220840 2024-03-22T19:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.092+0000 7f9593e00700 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a7220840 0x5628a7222c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.093+0000 7f957e000700 10 client.26365 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.093+0000 7f9593e00700 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a7220840 0x5628a7222c60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e487eeb89e772762 server_cookie=cafa1c48a8c8a726 in_seq=0 out_seq=0 2024-03-22T19:18:58.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.094+0000 7f957e000700 1 -- 172.21.15.86:0/1287295608 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5628a7155780 con 0x5628a7220840 2024-03-22T19:18:58.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.094+0000 7f957e000700 10 client.26365 handle_command_reply: tid=0 2024-03-22T19:18:58.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.094+0000 7f95ae9a5b80 1 client.26365 shutdown 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 msgr2=0x5628a72150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 0x5628a72150c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9584095610 tx=0x7f95840841a0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 shutdown_connections 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a7220840 0x5628a7222c60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a7214b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1287295608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a720b600 0x5628a72150c0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9584095610 tx=0x7f95840841a0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 >> 172.21.15.86:0/1287295608 conn(0x5628a71802a0 msgr2=0x5628a7205270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.095+0000 7f957e000700 0 client.26365 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.097+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 shutdown_connections 2024-03-22T19:18:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.097+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1287295608 wait complete. 2024-03-22T19:18:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.097+0000 7f95ae9a5b80 20 client.26365 trim_cache size 0 max 16384 2024-03-22T19:18:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.128+0000 7f95ae9a5b80 1 Processor -- start 2024-03-22T19:18:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.128+0000 7f95ae9a5b80 1 -- start start 2024-03-22T19:18:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.129+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a7214090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.129+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.129+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7214d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.129+0000 7f95ae9a5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a7200290 con 0x5628a720aca0 2024-03-22T19:18:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.129+0000 7f95ae9a5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a720b440 con 0x5628a720c4a0 2024-03-22T19:18:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.129+0000 7f95ae9a5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a7200000 con 0x5628a7214930 2024-03-22T19:18:58.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7214d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7214d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:43800/0 (socket says 172.21.15.86:43800) 2024-03-22T19:18:58.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 -- 172.21.15.86:0/1857712053 learned_addr learned my addr 172.21.15.86:0/1857712053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 -- 172.21.15.86:0/1857712053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 msgr2=0x5628a721c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 -- 172.21.15.86:0/1857712053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 msgr2=0x5628a7214090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a7214090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 -- 172.21.15.86:0/1857712053 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628a7122f20 con 0x5628a7214930 2024-03-22T19:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.131+0000 7f9598e00700 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7214d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f958000ab40 tx=0x7f958000ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c482728c8e565c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.132+0000 7f958a000700 1 -- 172.21.15.86:0/1857712053 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580012960 con 0x5628a7214930 2024-03-22T19:18:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.132+0000 7f958a000700 1 -- 172.21.15.86:0/1857712053 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f958000c040 con 0x5628a7214930 2024-03-22T19:18:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.132+0000 7f958a000700 1 -- 172.21.15.86:0/1857712053 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580014410 con 0x5628a7214930 2024-03-22T19:18:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1857712053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 msgr2=0x5628a7214d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7214d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f958000ab40 tx=0x7f958000ab70 comp rx=0 tx=0).stop 2024-03-22T19:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1857712053 shutdown_connections 2024-03-22T19:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7214d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a7214090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/1857712053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.134+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1857712053 >> 172.21.15.86:0/1857712053 conn(0x5628a71802a0 msgr2=0x5628a7212a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.135+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1857712053 shutdown_connections 2024-03-22T19:18:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.136+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/1857712053 wait complete. 2024-03-22T19:18:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.137+0000 7f95ae9a5b80 1 -- start start 2024-03-22T19:18:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.137+0000 7f95ae9a5b80 1 Processor -- start 2024-03-22T19:18:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.140+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a721b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.141+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.141+0000 7f9593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a721b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.142+0000 7f9593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a721b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45862/0 (socket says 172.21.15.86:45862) 2024-03-22T19:18:58.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.142+0000 7f95ae9a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7272900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.142+0000 7f9593400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.142+0000 7f9593400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44178/0 (socket says 172.21.15.86:44178) 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.142+0000 7f9598e00700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7272900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.142+0000 7f9593400700 1 -- 172.21.15.86:0/2268659529 learned_addr learned my addr 172.21.15.86:0/2268659529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.143+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a7272e40 con 0x5628a720aca0 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.143+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a7272f80 con 0x5628a720c4a0 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.143+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a7273110 con 0x5628a7214930 2024-03-22T19:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.144+0000 7f95ae9a5b80 10 client.0 _ll_register_callbacks cb 0x7f9584098ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.145+0000 7f9593e00700 1 -- 172.21.15.86:0/2268659529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 msgr2=0x5628a7272900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.145+0000 7f9593e00700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7272900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.145+0000 7f9593e00700 1 -- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 msgr2=0x5628a721bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.146+0000 7f9593e00700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.146+0000 7f9593e00700 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958000a7f0 con 0x5628a720aca0 2024-03-22T19:18:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.146+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f958c009480 con 0x5628a720aca0 2024-03-22T19:18:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.146+0000 7f9593400700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:18:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.146+0000 7f9598e00700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7272900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.146+0000 7f9593e00700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a721b6e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f958c007ab0 tx=0x7f958c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe49a0cb311925 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.147+0000 7f957e000700 10 client.16260 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.147+0000 7f957e000700 1 -- 172.21.15.86:0/2268659529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00e040 con 0x5628a720aca0 2024-03-22T19:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.147+0000 7f957e000700 1 -- 172.21.15.86:0/2268659529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f958c0340d0 con 0x5628a720aca0 2024-03-22T19:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.147+0000 7f957e000700 1 -- 172.21.15.86:0/2268659529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f958c0342f0 con 0x5628a720aca0 2024-03-22T19:18:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.148+0000 7f95ae9a5b80 10 client.16260 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.148+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=42}) v3 -- 0x5628a6fe2d50 con 0x5628a720aca0 2024-03-22T19:18:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.149+0000 7f957e000700 1 -- 172.21.15.86:0/2268659529 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f958c032480 con 0x5628a720aca0 2024-03-22T19:18:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f95ae9a5b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f95ae9a5b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f95ae9a5b80 20 client.16260 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a72743e0 0x5628a72767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f95ae9a5b80 4 client.16260 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-22T19:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5628a6fabd70 con 0x5628a72743e0 2024-03-22T19:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.153+0000 7f9593400700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a72743e0 0x5628a72767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.156+0000 7f9593400700 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a72743e0 0x5628a72767d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31ac17856c00fc6 server_cookie=bdcb6893f352c6a7 in_seq=0 out_seq=0 2024-03-22T19:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f957e000700 10 client.16260 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f957e000700 1 -- 172.21.15.86:0/2268659529 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5628a6fabd70 con 0x5628a72743e0 2024-03-22T19:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f957e000700 10 client.16260 handle_command_reply: tid=0 2024-03-22T19:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 client.16260 shutdown 2024-03-22T19:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 msgr2=0x5628a721b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a721b6e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f958c007ab0 tx=0x7f958c00b8f0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 shutdown_connections 2024-03-22T19:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5628a72743e0 0x5628a72767d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628a7214930 0x5628a7272900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628a720aca0 0x5628a721b6e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 --2- 172.21.15.86:0/2268659529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628a720c4a0 0x5628a721bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f957e000700 0 client.16260 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.159+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 >> 172.21.15.86:0/2268659529 conn(0x5628a71802a0 msgr2=0x5628a7214d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.160+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 shutdown_connections 2024-03-22T19:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.160+0000 7f95ae9a5b80 1 -- 172.21.15.86:0/2268659529 wait complete. 2024-03-22T19:18:58.163 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:18:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T19:18:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "6c3c3885-e297-428b-887d-87990c703008", 2024-03-22T19:18:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T19:18:58.163 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.161+0000 7f95ae9a5b80 20 client.16260 trim_cache size 0 max 16384 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95780a0be0 msgr2=0x7f95780a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95780a0be0 0x7f95780a30a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b070 tx=0x7f958c007b80 comp rx=0 tx=0).stop 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 msgr2=0x7f95941a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 0x7f95941a6230 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f958400b530 tx=0x7f958400b8f0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 shutdown_connections 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f95780a0be0 0x7f95780a30a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9594108ee0 0x7f95941ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95940721e0 0x7f95941a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 --2- 172.21.15.86:0/4242150909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594108520 0x7f95941a6230 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 >> 172.21.15.86:0/4242150909 conn(0x7f959406dd20 msgr2=0x7f959406f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.162+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 shutdown_connections 2024-03-22T19:18:58.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:58.163+0000 7f959ae00700 1 -- 172.21.15.86:0/4242150909 wait complete. 2024-03-22T19:18:58.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:18:58.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:58 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:18:58.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:58 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:18:58.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:58 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:18:59.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.320+0000 7fec7ea00700 1 -- 172.21.15.86:0/3246508574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810c930 msgr2=0x7fec7810cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.320+0000 7fec7ea00700 1 --2- 172.21.15.86:0/3246508574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810c930 0x7fec7810cd90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec600099c0 tx=0x7fec60009cd0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 -- 172.21.15.86:0/3246508574 shutdown_connections 2024-03-22T19:18:59.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 --2- 172.21.15.86:0/3246508574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7810d2d0 0x7fec78113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 --2- 172.21.15.86:0/3246508574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810b740 0x7fec7810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 --2- 172.21.15.86:0/3246508574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810c930 0x7fec7810cd90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 -- 172.21.15.86:0/3246508574 >> 172.21.15.86:0/3246508574 conn(0x7fec7806b6f0 msgr2=0x7fec7806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 -- 172.21.15.86:0/3246508574 shutdown_connections 2024-03-22T19:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.321+0000 7fec7ea00700 1 -- 172.21.15.86:0/3246508574 wait complete. 2024-03-22T19:18:59.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.323+0000 7fec7ea00700 1 Processor -- start 2024-03-22T19:18:59.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 -- start start 2024-03-22T19:18:59.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810b740 0x7fec781a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 0x7fec781a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7810d2d0 0x7fec781a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781a9250 con 0x7fec7810d2d0 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781a9390 con 0x7fec7810b740 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.324+0000 7fec7ea00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781a9520 con 0x7fec7810c930 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.325+0000 7fec6f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 0x7fec781a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.325+0000 7fec77e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810b740 0x7fec781a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.325+0000 7fec6f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 0x7fec781a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:45998/0 (socket says 172.21.15.86:45998) 2024-03-22T19:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.325+0000 7fec6f400700 1 -- 172.21.15.86:0/1095548775 learned_addr learned my addr 172.21.15.86:0/1095548775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.326+0000 7fec6f400700 1 -- 172.21.15.86:0/1095548775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810b740 msgr2=0x7fec781a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.326+0000 7fec6f400700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810b740 0x7fec781a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.326+0000 7fec6f400700 1 -- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7810d2d0 msgr2=0x7fec781a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.326+0000 7fec6f400700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7810d2d0 0x7fec781a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.326+0000 7fec6f400700 1 -- 172.21.15.86:0/1095548775 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600095f0 con 0x7fec7810c930 2024-03-22T19:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.326+0000 7fec6f400700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 0x7fec781a1c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec60003730 tx=0x7fec6000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.327+0000 7fec75600700 1 -- 172.21.15.86:0/1095548775 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6001d070 con 0x7fec7810c930 2024-03-22T19:18:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.327+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec781a97a0 con 0x7fec7810c930 2024-03-22T19:18:59.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.327+0000 7fec75600700 1 -- 172.21.15.86:0/1095548775 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6000fe70 con 0x7fec7810c930 2024-03-22T19:18:59.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.327+0000 7fec75600700 1 -- 172.21.15.86:0/1095548775 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60017650 con 0x7fec7810c930 2024-03-22T19:18:59.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.327+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec781a9b60 con 0x7fec7810c930 2024-03-22T19:18:59.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.336+0000 7fec75600700 1 -- 172.21.15.86:0/1095548775 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fec600177b0 con 0x7fec7810c930 2024-03-22T19:18:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.340+0000 7fec75600700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fec5c0a0b10 0x7fec5c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.340+0000 7fec75600700 1 -- 172.21.15.86:0/1095548775 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fec600cd260 con 0x7fec7810c930 2024-03-22T19:18:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.341+0000 7fec77e00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fec5c0a0b10 0x7fec5c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.341+0000 7fec924e1b80 1 Processor -- start 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 -- start start 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec77e00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fec5c0a0b10 0x7fec5c0a2fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fec7810c790 tx=0x7fec68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x5577617cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x5577617cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776171b990 con 0x5577617ccc00 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557761719350 con 0x5577617cb3b0 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec924e1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557761714c40 con 0x5577617cbd10 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.342+0000 7fec6f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x5577617cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x5577617cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:46012/0 (socket says 172.21.15.86:46012) 2024-03-22T19:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 -- 172.21.15.86:0/3107820618 learned_addr learned my addr 172.21.15.86:0/3107820618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec77e00700 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec7ca00700 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x5577617cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 -- 172.21.15.86:0/3107820618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 msgr2=0x5577617cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 -- 172.21.15.86:0/3107820618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 msgr2=0x5577617cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x5577617cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.343+0000 7fec6f400700 1 -- 172.21.15.86:0/3107820618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577615963b0 con 0x5577617cbd10 2024-03-22T19:18:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.344+0000 7fec6f400700 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x5577617cc670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec6009ae80 tx=0x7fec60009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b15c9dd3a98b9cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:59.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.348+0000 7fec6d200700 1 -- 172.21.15.86:0/3107820618 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6009a850 con 0x5577617cbd10 2024-03-22T19:18:59.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.348+0000 7fec6d200700 1 -- 172.21.15.86:0/3107820618 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6009b5f0 con 0x5577617cbd10 2024-03-22T19:18:59.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.348+0000 7fec6d200700 1 -- 172.21.15.86:0/3107820618 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6009a640 con 0x5577617cbd10 2024-03-22T19:18:59.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.351+0000 7fec924e1b80 1 -- 172.21.15.86:0/3107820618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 msgr2=0x5577617cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.351+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x5577617cc670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec6009ae80 tx=0x7fec60009310 comp rx=0 tx=0).stop 2024-03-22T19:18:59.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 -- 172.21.15.86:0/3107820618 shutdown_connections 2024-03-22T19:18:59.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x5577617cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3107820618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x5577617cc670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 -- 172.21.15.86:0/3107820618 >> 172.21.15.86:0/3107820618 conn(0x557761738220 msgr2=0x5577617bd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:59.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 -- 172.21.15.86:0/3107820618 shutdown_connections 2024-03-22T19:18:59.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.353+0000 7fec924e1b80 1 -- 172.21.15.86:0/3107820618 wait complete. 2024-03-22T19:18:59.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.355+0000 7fec924e1b80 1 -- start start 2024-03-22T19:18:59.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.355+0000 7fec924e1b80 1 Processor -- start 2024-03-22T19:18:59.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.356+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.356+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cbd10 0x5577617d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.356+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.357+0000 7fec77e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.357+0000 7fec77e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:54450/0 (socket says 172.21.15.86:54450) 2024-03-22T19:18:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec7ca00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec7ca00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617dc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:46022/0 (socket says 172.21.15.86:46022) 2024-03-22T19:18:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec7ca00700 1 -- 172.21.15.86:0/3062609659 learned_addr learned my addr 172.21.15.86:0/3062609659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:59.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec6f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cbd10 0x5577617d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577616b0d80 con 0x5577617cbd10 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577616b0a20 con 0x5577617cb3b0 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.358+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577615986a0 con 0x5577617ccc00 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.359+0000 7fec924e1b80 10 client.0 _ll_register_callbacks cb 0x55776151c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.360+0000 7fec6f400700 1 -- 172.21.15.86:0/3062609659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 msgr2=0x5577617dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.360+0000 7fec6f400700 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.360+0000 7fec6f400700 1 -- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 msgr2=0x5577617d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.360+0000 7fec6f400700 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.360+0000 7fec6f400700 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec781a9b60 con 0x5577617cbd10 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.360+0000 7fec6f400700 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cbd10 0x5577617d5540 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5577617cc110 tx=0x7fec600cad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9e46e3100a06f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.361+0000 7fec65600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.361+0000 7fec65600700 1 -- 172.21.15.86:0/3062609659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600a3050 con 0x5577617cbd10 2024-03-22T19:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.361+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577617b5ad0 con 0x5577617cbd10 2024-03-22T19:18:59.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.361+0000 7fec65600700 1 -- 172.21.15.86:0/3062609659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fec6009aef0 con 0x5577617cbd10 2024-03-22T19:18:59.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.361+0000 7fec65600700 1 -- 172.21.15.86:0/3062609659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec600b3360 con 0x5577617cbd10 2024-03-22T19:18:59.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 10 client.16266 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557761542b60 con 0x5577617cbd10 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec65600700 1 -- 172.21.15.86:0/3062609659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577617b5ad0 con 0x5577617cbd10 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec65600700 1 -- 172.21.15.86:0/3062609659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fec6007d050 con 0x5577617cbd10 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 10 client.16266 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 20 client.16266 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5577617e0850 0x5577617e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 4 client.16266 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.362+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55776170d700 con 0x5577617e0850 2024-03-22T19:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.364+0000 7fec77e00700 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5577617e0850 0x5577617e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.366+0000 7fec77e00700 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5577617e0850 0x5577617e2c70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547c871c2524c60 server_cookie=1fc2b064ce8b0e6d in_seq=0 out_seq=0 2024-03-22T19:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.366+0000 7fec65600700 10 client.16266 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.367+0000 7fec65600700 1 -- 172.21.15.86:0/3062609659 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55776170d700 con 0x5577617e0850 2024-03-22T19:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.367+0000 7fec65600700 10 client.16266 handle_command_reply: tid=0 2024-03-22T19:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.367+0000 7fec924e1b80 1 client.16266 shutdown 2024-03-22T19:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cbd10 msgr2=0x5577617d5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cbd10 0x5577617d5540 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5577617cc110 tx=0x7fec600cad60 comp rx=0 tx=0).stop 2024-03-22T19:18:59.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 shutdown_connections 2024-03-22T19:18:59.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5577617e0850 0x5577617e2c70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x5577617d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec65600700 0 client.16266 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 --2- 172.21.15.86:0/3062609659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cbd10 0x5577617d5540 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 >> 172.21.15.86:0/3062609659 conn(0x557761738220 msgr2=0x5577617ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 shutdown_connections 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.368+0000 7fec924e1b80 1 -- 172.21.15.86:0/3062609659 wait complete. 2024-03-22T19:18:59.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.369+0000 7fec924e1b80 20 client.16266 trim_cache size 0 max 16384 2024-03-22T19:18:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.411+0000 7fec924e1b80 1 Processor -- start 2024-03-22T19:18:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 -- start start 2024-03-22T19:18:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cb3b0 0x5577617dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 0x5577617da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557761788940 con 0x5577617cb3b0 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577617d5f00 con 0x5577617cbd10 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec924e1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577617d4b70 con 0x5577617ccc00 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.412+0000 7fec6f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 0x5577617da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 0x5577617da530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:54466/0 (socket says 172.21.15.86:54466) 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 -- 172.21.15.86:0/4036195265 learned_addr learned my addr 172.21.15.86:0/4036195265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec77e00700 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cb3b0 0x5577617dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 -- 172.21.15.86:0/4036195265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 msgr2=0x5577617ccfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617ccfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 -- 172.21.15.86:0/4036195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cb3b0 msgr2=0x5577617dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cb3b0 0x5577617dcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 -- 172.21.15.86:0/4036195265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577616daea0 con 0x5577617cbd10 2024-03-22T19:18:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.413+0000 7fec6f400700 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 0x5577617da530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec60000c00 tx=0x7fec60097160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb9bb4978897e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:59.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.414+0000 7fec6dc00700 1 -- 172.21.15.86:0/4036195265 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600a3050 con 0x5577617cbd10 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.415+0000 7fec6dc00700 1 -- 172.21.15.86:0/4036195265 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec600b3360 con 0x5577617cbd10 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.415+0000 7fec6dc00700 1 -- 172.21.15.86:0/4036195265 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600ba2d0 con 0x5577617cbd10 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 -- 172.21.15.86:0/4036195265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 msgr2=0x5577617da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 0x5577617da530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec60000c00 tx=0x7fec60097160 comp rx=0 tx=0).stop 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 -- 172.21.15.86:0/4036195265 shutdown_connections 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617ccc00 0x5577617ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617cb3b0 0x5577617dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 --2- 172.21.15.86:0/4036195265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cbd10 0x5577617da530 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 -- 172.21.15.86:0/4036195265 >> 172.21.15.86:0/4036195265 conn(0x557761738220 msgr2=0x5577617b9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.416+0000 7fec924e1b80 1 -- 172.21.15.86:0/4036195265 shutdown_connections 2024-03-22T19:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.417+0000 7fec924e1b80 1 -- 172.21.15.86:0/4036195265 wait complete. 2024-03-22T19:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.417+0000 7fec924e1b80 1 -- start start 2024-03-22T19:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.418+0000 7fec924e1b80 1 Processor -- start 2024-03-22T19:18:59.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.420+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x55776182a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.420+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x55776182aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.420+0000 7fec924e1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x557761831ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.420+0000 7fec77e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x55776182a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.420+0000 7fec77e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x55776182a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:54470/0 (socket says 172.21.15.86:54470) 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec6f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x55776182aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec6f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x55776182aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:46038/0 (socket says 172.21.15.86:46038) 2024-03-22T19:18:59.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec924e1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557761832000 con 0x5577617ccc00 2024-03-22T19:18:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec924e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557761832140 con 0x5577617cb3b0 2024-03-22T19:18:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec924e1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577618322d0 con 0x5577617cbd10 2024-03-22T19:18:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec924e1b80 10 client.0 _ll_register_callbacks cb 0x7fec68011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:18:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.421+0000 7fec77e00700 1 -- 172.21.15.86:0/2831054935 learned_addr learned my addr 172.21.15.86:0/2831054935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.422+0000 7fec7ca00700 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x557761831ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.423+0000 7fec77e00700 1 -- 172.21.15.86:0/2831054935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 msgr2=0x55776182aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.423+0000 7fec77e00700 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x55776182aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.423+0000 7fec77e00700 1 -- 172.21.15.86:0/2831054935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 msgr2=0x557761831ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.423+0000 7fec77e00700 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x557761831ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.423+0000 7fec77e00700 1 -- 172.21.15.86:0/2831054935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557761542b60 con 0x5577617cb3b0 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.424+0000 7fec77e00700 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x55776182a570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5577617e3550 tx=0x7fec680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec99a945860d436b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:18:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.424+0000 7fec65600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:18:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.425+0000 7fec65600700 1 -- 172.21.15.86:0/2831054935 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68022040 con 0x5577617cb3b0 2024-03-22T19:18:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.425+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec6009b5f0 con 0x5577617cb3b0 2024-03-22T19:18:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.425+0000 7fec65600700 1 -- 172.21.15.86:0/2831054935 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fec68004020 con 0x5577617cb3b0 2024-03-22T19:18:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.426+0000 7fec65600700 1 -- 172.21.15.86:0/2831054935 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec680041e0 con 0x5577617cb3b0 2024-03-22T19:18:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.426+0000 7fec924e1b80 10 client.26399 fetch_fsmap learned FSMap version 42 2024-03-22T19:18:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.426+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55776159acd0 con 0x5577617cb3b0 2024-03-22T19:18:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.426+0000 7fec65600700 1 -- 172.21.15.86:0/2831054935 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6009b5f0 con 0x5577617cb3b0 2024-03-22T19:18:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec65600700 1 -- 172.21.15.86:0/2831054935 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fec68033440 con 0x5577617cb3b0 2024-03-22T19:18:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec924e1b80 10 client.26399 fetch_fsmap finished waiting for FSMap version 42 2024-03-22T19:18:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec924e1b80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:18:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec924e1b80 20 client.26399 populate_metadata read hostname 'smithi086' 2024-03-22T19:18:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec924e1b80 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557761833d40 0x557761836160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:18:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec924e1b80 4 client.26399 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:18:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.428+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557761563cf0 con 0x557761833d40 2024-03-22T19:18:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.431+0000 7fec6f400700 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557761833d40 0x557761836160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:18:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.432+0000 7fec65600700 10 client.26399 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.432+0000 7fec6f400700 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557761833d40 0x557761836160 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3d34730f88a48f server_cookie=7b19cf8dcaad6f90 in_seq=0 out_seq=0 2024-03-22T19:18:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.433+0000 7fec65600700 1 -- 172.21.15.86:0/2831054935 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x557761563cf0 con 0x557761833d40 2024-03-22T19:18:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.433+0000 7fec65600700 10 client.26399 handle_command_reply: tid=0 2024-03-22T19:18:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.433+0000 7fec924e1b80 1 client.26399 shutdown 2024-03-22T19:18:59.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.433+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 msgr2=0x55776182a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.433+0000 7fec924e1b80 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x55776182a570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5577617e3550 tx=0x7fec680047c0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.439 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:18:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (106 inodes in the stack)", 2024-03-22T19:18:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "6c3c3885-e297-428b-887d-87990c703008": { 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "6c3c3885-e297-428b-887d-87990c703008", 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:18:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 shutdown_connections 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x557761833d40 0x557761836160 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577617ccc00 0x557761831ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577617cb3b0 0x55776182a570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 --2- 172.21.15.86:0/2831054935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577617cbd10 0x55776182aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec65600700 0 client.26399 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 >> 172.21.15.86:0/2831054935 conn(0x557761738220 msgr2=0x5577617bc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 shutdown_connections 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 1 -- 172.21.15.86:0/2831054935 wait complete. 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.436+0000 7fec924e1b80 20 client.26399 trim_cache size 0 max 16384 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fec5c0a0b10 msgr2=0x7fec5c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fec5c0a0b10 0x7fec5c0a2fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fec7810c790 tx=0x7fec68009260 comp rx=0 tx=0).stop 2024-03-22T19:18:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 msgr2=0x7fec781a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 0x7fec781a1c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec60003730 tx=0x7fec6000fa00 comp rx=0 tx=0).stop 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 shutdown_connections 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fec5c0a0b10 0x7fec5c0a2fd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7810d2d0 0x7fec781a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7810b740 0x7fec781a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 --2- 172.21.15.86:0/1095548775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec7810c930 0x7fec781a1c60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:18:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 >> 172.21.15.86:0/1095548775 conn(0x7fec7806b6f0 msgr2=0x7fec7810fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:18:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.438+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 shutdown_connections 2024-03-22T19:18:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:18:59.439+0000 7fec7ea00700 1 -- 172.21.15.86:0/1095548775 wait complete. 2024-03-22T19:18:59.482 INFO:tasks.cephfs.filesystem:scrub status for tag:6c3c3885-e297-428b-887d-87990c703008 - {'path': '/', 'tag': '6c3c3885-e297-428b-887d-87990c703008', 'options': 'recursive,force'} 2024-03-22T19:18:59.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: scrub queued for path: / 2024-03-22T19:18:59.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:59.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:18:59.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:18:59.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:18:59.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:18:59.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:18:59 smithi131 ceph-mon[78559]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 70 MiB/s wr, 13.07k op/s 2024-03-22T19:18:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: scrub queued for path: / 2024-03-22T19:18:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:18:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:18:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:18:59.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:18:59.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:18:59 smithi086 ceph-mon[38032]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 70 MiB/s wr, 13.07k op/s 2024-03-22T19:18:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: scrub queued for path: / 2024-03-22T19:18:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:18:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:18:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:18:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:18:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:18:59.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:18:59 smithi167 ceph-mon[47728]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 70 MiB/s wr, 13.07k op/s 2024-03-22T19:19:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:01 smithi086 ceph-mon[38032]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 72 MiB/s wr, 13.53k op/s 2024-03-22T19:19:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:01 smithi167 ceph-mon[47728]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 72 MiB/s wr, 13.53k op/s 2024-03-22T19:19:01.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:01 smithi131 ceph-mon[78559]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 72 MiB/s wr, 13.53k op/s 2024-03-22T19:19:03.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:03 smithi086 ceph-mon[38032]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 67 MiB/s wr, 12.62k op/s 2024-03-22T19:19:03.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:03 smithi167 ceph-mon[47728]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 67 MiB/s wr, 12.62k op/s 2024-03-22T19:19:03.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:03 smithi131 ceph-mon[78559]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 67 MiB/s wr, 12.62k op/s 2024-03-22T19:19:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:05 smithi086 ceph-mon[38032]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 52 MiB/s wr, 10.01k op/s 2024-03-22T19:19:05.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:05 smithi167 ceph-mon[47728]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 52 MiB/s wr, 10.01k op/s 2024-03-22T19:19:05.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:05 smithi131 ceph-mon[78559]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 52 MiB/s wr, 10.01k op/s 2024-03-22T19:19:07.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:07 smithi131 ceph-mon[78559]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 51 MiB/s wr, 10.10k op/s 2024-03-22T19:19:07.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:07 smithi086 ceph-mon[38032]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 51 MiB/s wr, 10.10k op/s 2024-03-22T19:19:07.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:07 smithi167 ceph-mon[47728]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 51 MiB/s wr, 10.10k op/s 2024-03-22T19:19:08.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:08 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:08.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:08 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:08 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:08.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:08 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:08 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:08.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:08 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:09.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:09 smithi131 ceph-mon[78559]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 46 MiB/s wr, 8.79k op/s 2024-03-22T19:19:09.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:09 smithi086 ceph-mon[38032]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 46 MiB/s wr, 8.79k op/s 2024-03-22T19:19:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:09 smithi167 ceph-mon[47728]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 46 MiB/s wr, 8.79k op/s 2024-03-22T19:19:10.558 INFO:tasks.workunit.client.0.smithi086.stdout:Syncing()...36 sec 2024-03-22T19:19:10.558 INFO:tasks.workunit.client.0.smithi086.stdout:FFSB benchmark finished at: Fri Mar 22 19:19:10 2024 2024-03-22T19:19:10.558 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.558 INFO:tasks.workunit.client.0.smithi086.stdout:Results: 2024-03-22T19:19:10.558 INFO:tasks.workunit.client.0.smithi086.stdout:Benchmark took 342.01 sec 2024-03-22T19:19:10.558 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:Total Results 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:=============== 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout: ======= ============ ========= ======= =========== ========== 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout: write : 10885120 31826.64 100.000% 100.000% 124MB/sec 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:- 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:31826.64 Transactions per Second 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:Throughput Results 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:=================== 2024-03-22T19:19:10.559 INFO:tasks.workunit.client.0.smithi086.stdout:Write Throughput: 124MB/sec 2024-03-22T19:19:10.658 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.658 INFO:tasks.workunit.client.0.smithi086.stdout:System Call Latency statistics in millisecs 2024-03-22T19:19:10.658 INFO:tasks.workunit.client.0.smithi086.stdout:===== 2024-03-22T19:19:10.658 INFO:tasks.workunit.client.0.smithi086.stdout: Min Avg Max Total Calls 2024-03-22T19:19:10.658 INFO:tasks.workunit.client.0.smithi086.stdout: ======== ======== ======== ============ 2024-03-22T19:19:10.658 INFO:tasks.workunit.client.0.smithi086.stdout:[ open] 0.017000 2.483592 11327.827148 8504 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[0] 5000.000000 - 10000.000000 : 1 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[7] 20.000000 - 50.000000 : 7 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[8] 10.000000 - 20.000000 : 30 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[9] 5.000000 - 10.000000 : 68 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[10] 2.000000 - 5.000000 : 153 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[11] 1.000000 - 2.000000 : 93 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[12] 0.500000 - 1.000000 : 99 2024-03-22T19:19:10.659 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[13] 0.200000 - 0.500000 : 113 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[14] 0.100000 - 0.200000 : 952 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[15] 0.050000 - 0.100000 : 595 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[16] 0.020000 - 0.050000 : 6386 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[17] 0.010000 - 0.020000 : 5 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout:[ write] 0.003000 0.889455 62100.027344 10885120 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[0] 5000.000000 - 10000.000000 : 244 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[1] 2000.000000 - 5000.000000 : 472 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[2] 1000.000000 - 2000.000000 : 445 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[3] 500.000000 - 1000.000000 : 376 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[4] 200.000000 - 500.000000 : 462 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[5] 100.000000 - 200.000000 : 398 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[6] 50.000000 - 100.000000 : 423 2024-03-22T19:19:10.660 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[7] 20.000000 - 50.000000 : 1162 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[8] 10.000000 - 20.000000 : 2549 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[9] 5.000000 - 10.000000 : 3830 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[10] 2.000000 - 5.000000 : 6901 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[11] 1.000000 - 2.000000 : 3830 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[12] 0.500000 - 1.000000 : 3402 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[13] 0.200000 - 0.500000 : 5630 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[14] 0.100000 - 0.200000 : 8762 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[15] 0.050000 - 0.100000 : 29037 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[16] 0.020000 - 0.050000 : 239312 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[17] 0.010000 - 0.020000 : 3626740 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[18] 0.000000 - 0.010000 : 6950843 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout:[ lseek] 0.000000 0.001933 49.013000 10885120 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-03-22T19:19:10.661 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[7] 20.000000 - 50.000000 : 64 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[8] 10.000000 - 20.000000 : 201 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[9] 5.000000 - 10.000000 : 299 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[10] 2.000000 - 5.000000 : 612 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[11] 1.000000 - 2.000000 : 353 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[12] 0.500000 - 1.000000 : 296 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[13] 0.200000 - 0.500000 : 470 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[14] 0.100000 - 0.200000 : 509 2024-03-22T19:19:10.662 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[15] 0.050000 - 0.100000 : 1422 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[16] 0.020000 - 0.050000 : 5555 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[17] 0.010000 - 0.020000 : 4673 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[18] 0.000000 - 0.010000 : 10870666 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout:[ close] 0.003000 0.014202 13.022000 8504 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-03-22T19:19:10.663 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[8] 10.000000 - 20.000000 : 1 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[9] 5.000000 - 10.000000 : 2 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[10] 2.000000 - 5.000000 : 3 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[11] 1.000000 - 2.000000 : 2 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[12] 0.500000 - 1.000000 : 4 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[13] 0.200000 - 0.500000 : 9 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[14] 0.100000 - 0.200000 : 4 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[15] 0.050000 - 0.100000 : 15 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[16] 0.020000 - 0.050000 : 34 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[17] 0.010000 - 0.020000 : 3267 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: msec_range[18] 0.000000 - 0.010000 : 5163 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.664 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.665 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-22T19:19:10.665 INFO:tasks.workunit.client.0.smithi086.stdout:2.1% User Time 2024-03-22T19:19:10.665 INFO:tasks.workunit.client.0.smithi086.stdout:34.0% System Time 2024-03-22T19:19:10.665 INFO:tasks.workunit.client.0.smithi086.stdout:36.1% CPU Utilization 2024-03-22T19:19:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:10 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:10.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:10 smithi131 ceph-mon[78559]: scrub summary: active paths [/] 2024-03-22T19:19:10.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:10 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:10.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:10 smithi086 ceph-mon[38032]: scrub summary: active paths [/] 2024-03-22T19:19:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:10 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:10 smithi167 ceph-mon[47728]: scrub summary: active paths [/] 2024-03-22T19:19:11.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:11 smithi131 ceph-mon[78559]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 8.85k op/s 2024-03-22T19:19:11.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:11 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:11.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:11 smithi086 ceph-mon[38032]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 8.85k op/s 2024-03-22T19:19:11.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:11 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:11 smithi167 ceph-mon[47728]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 8.85k op/s 2024-03-22T19:19:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:11 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:12.663 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T19:19:12.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-22T19:19:12.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:12 smithi086 ceph-mon[38032]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:12.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:12 smithi131 ceph-mon[78559]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:12.737 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-03-22T19:19:12.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-22T19:19:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:12 smithi167 ceph-mon[47728]: scrub summary: idle+waiting paths [/] 2024-03-22T19:19:13.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:13 smithi086 ceph-mon[38032]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 37 MiB/s wr, 7.25k op/s 2024-03-22T19:19:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:13 smithi167 ceph-mon[47728]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 37 MiB/s wr, 7.25k op/s 2024-03-22T19:19:13.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:13 smithi131 ceph-mon[78559]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 37 MiB/s wr, 7.25k op/s 2024-03-22T19:19:14.214 DEBUG:teuthology.parallel:result is None 2024-03-22T19:19:14.214 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T19:19:14.357 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T19:19:14.357 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-22T19:19:14.369 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-22T19:19:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:14 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:14 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:14.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:14 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:15 smithi086 ceph-mon[38032]: pgmap v534: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 35 MiB/s wr, 6.95k op/s 2024-03-22T19:19:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:15 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:15 smithi167 ceph-mon[47728]: pgmap v534: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 35 MiB/s wr, 6.95k op/s 2024-03-22T19:19:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:15 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:15.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:15 smithi131 ceph-mon[78559]: pgmap v534: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 35 MiB/s wr, 6.95k op/s 2024-03-22T19:19:15.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:15 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:16.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:16 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:16.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:16 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:17.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:16 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:17.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:17 smithi167 ceph-mon[47728]: pgmap v535: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 32 MiB/s wr, 6.34k op/s 2024-03-22T19:19:17.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:17 smithi167 ceph-mon[47728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:17.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:17 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:19:17.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:17 smithi131 ceph-mon[78559]: pgmap v535: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 32 MiB/s wr, 6.34k op/s 2024-03-22T19:19:17.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:17 smithi131 ceph-mon[78559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:17.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:17 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:19:18.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:17 smithi086 ceph-mon[38032]: pgmap v535: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 32 MiB/s wr, 6.34k op/s 2024-03-22T19:19:18.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:17 smithi086 ceph-mon[38032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-22T19:19:18.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:17 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:19:18.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:19:18.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:19:18.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:19:18.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:18 smithi167 ceph-mon[47728]: scrub summary: idle 2024-03-22T19:19:18.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:19:18.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:19:18.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:19:18.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:18 smithi131 ceph-mon[78559]: scrub summary: idle 2024-03-22T19:19:19.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:19:19.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:19:19.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:19:19.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:18 smithi086 ceph-mon[38032]: scrub summary: idle 2024-03-22T19:19:19.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:19 smithi167 ceph-mon[47728]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 21 MiB/s wr, 4.34k op/s 2024-03-22T19:19:19.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:19 smithi131 ceph-mon[78559]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 21 MiB/s wr, 4.34k op/s 2024-03-22T19:19:20.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:19 smithi086 ceph-mon[38032]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 21 MiB/s wr, 4.34k op/s 2024-03-22T19:19:21.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:21 smithi167 ceph-mon[47728]: pgmap v537: 97 pgs: 97 active+clean; 8.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 2.91k op/s 2024-03-22T19:19:21.974 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:21 smithi131 ceph-mon[78559]: pgmap v537: 97 pgs: 97 active+clean; 8.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 2.91k op/s 2024-03-22T19:19:22.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:21 smithi086 ceph-mon[38032]: pgmap v537: 97 pgs: 97 active+clean; 8.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 2.91k op/s 2024-03-22T19:19:23.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:23 smithi131 ceph-mon[78559]: pgmap v538: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.7 MiB/s wr, 1.52k op/s 2024-03-22T19:19:24.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:23 smithi086 ceph-mon[38032]: pgmap v538: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.7 MiB/s wr, 1.52k op/s 2024-03-22T19:19:24.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:23 smithi167 ceph-mon[47728]: pgmap v538: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.7 MiB/s wr, 1.52k op/s 2024-03-22T19:19:25.971 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:25 smithi131 ceph-mon[78559]: pgmap v539: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.7 MiB/s wr, 1.56k op/s 2024-03-22T19:19:26.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:25 smithi086 ceph-mon[38032]: pgmap v539: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.7 MiB/s wr, 1.56k op/s 2024-03-22T19:19:26.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:25 smithi167 ceph-mon[47728]: pgmap v539: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.7 MiB/s wr, 1.56k op/s 2024-03-22T19:19:27.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:27 smithi131 ceph-mon[78559]: pgmap v540: 97 pgs: 97 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 95 KiB/s wr, 233 op/s 2024-03-22T19:19:27.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:27 smithi086 ceph-mon[38032]: pgmap v540: 97 pgs: 97 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 95 KiB/s wr, 233 op/s 2024-03-22T19:19:27.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:27 smithi167 ceph-mon[47728]: pgmap v540: 97 pgs: 97 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 95 KiB/s wr, 233 op/s 2024-03-22T19:19:29.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-22T19:19:29.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 -- 172.21.15.86:0/2881911554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f64101f30 msgr2=0x7f9f64102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2881911554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f64101f30 0x7f9f64102390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009920 tx=0x7f9f50009c30 comp rx=0 tx=0).stop 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 -- 172.21.15.86:0/2881911554 shutdown_connections 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2881911554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f641028d0 0x7f9f64109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2881911554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2881911554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f64101f30 0x7f9f64102390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 -- 172.21.15.86:0/2881911554 >> 172.21.15.86:0/2881911554 conn(0x7f9f640fc2b0 msgr2=0x7f9f640fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.682+0000 7f9f6a400700 1 -- 172.21.15.86:0/2881911554 shutdown_connections 2024-03-22T19:19:29.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.683+0000 7f9f6a400700 1 -- 172.21.15.86:0/2881911554 wait complete. 2024-03-22T19:19:29.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.684+0000 7f9f6a400700 1 Processor -- start 2024-03-22T19:19:29.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.684+0000 7f9f6a400700 1 -- start start 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f6a400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f6a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f64101f30 0x7f9f640731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f6a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641028d0 0x7f9f640736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f6a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64073de0 con 0x7f9f64101f30 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f6a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64073f50 con 0x7f9f641028d0 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f6a400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641a0de0 con 0x7f9f64100d40 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f63400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f63400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42422/0 (socket says 172.21.15.86:42422) 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f63400700 1 -- 172.21.15.86:0/2165783740 learned_addr learned my addr 172.21.15.86:0/2165783740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f63e00700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641028d0 0x7f9f640736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.685+0000 7f9f62a00700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f64101f30 0x7f9f640731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f63400700 1 -- 172.21.15.86:0/2165783740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641028d0 msgr2=0x7f9f640736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f63400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641028d0 0x7f9f640736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f63400700 1 -- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f64101f30 msgr2=0x7f9f640731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f63400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f64101f30 0x7f9f640731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f63400700 1 -- 172.21.15.86:0/2165783740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009520 con 0x7f9f64100d40 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f63400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64074ac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e970 tx=0x7f9f5400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f5be00700 1 -- 172.21.15.86:0/2165783740 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54009790 con 0x7f9f64100d40 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f5be00700 1 -- 172.21.15.86:0/2165783740 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5400c7e0 con 0x7f9f64100d40 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f641a0fe0 con 0x7f9f64100d40 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f5be00700 1 -- 172.21.15.86:0/2165783740 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003c70 con 0x7f9f64100d40 2024-03-22T19:19:29.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.686+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641a1460 con 0x7f9f64100d40 2024-03-22T19:19:29.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.689+0000 7f9f5be00700 1 -- 172.21.15.86:0/2165783740 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f9f5400cc50 con 0x7f9f64100d40 2024-03-22T19:19:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 Processor -- start 2024-03-22T19:19:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 -- start start 2024-03-22T19:19:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04dc20 0x5627ee04e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04e580 0x5627ee04ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee04f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627edfa6990 con 0x5627ee04dc20 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627edfa4350 con 0x5627ee04e580 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627edf9fc40 con 0x5627ee04f420 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.691+0000 7f9f5be00700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f480a0a90 0x7f9f480a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04e580 0x5627ee04ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04dc20 0x5627ee04e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04dc20 0x5627ee04e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37082/0 (socket says 172.21.15.86:37082) 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63e00700 1 -- 172.21.15.86:0/3510845619 learned_addr learned my addr 172.21.15.86:0/3510845619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f62a00700 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee04f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63e00700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f480a0a90 0x7f9f480a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f5be00700 1 -- 172.21.15.86:0/2165783740 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9f54014070 con 0x7f9f64100d40 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 -- 172.21.15.86:0/3510845619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 msgr2=0x5627ee04f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee04f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 -- 172.21.15.86:0/3510845619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04dc20 msgr2=0x5627ee04e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04dc20 0x5627ee04e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 -- 172.21.15.86:0/3510845619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627ede213b0 con 0x5627ee04e580 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63400700 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04e580 0x5627ee04ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f54099820 tx=0x7f9f54007b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63647b6b26351c20 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.692+0000 7f9f63e00700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f480a0a90 0x7f9f480a2f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009fd0 tx=0x7f9f5c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f58c00700 1 -- 172.21.15.86:0/3510845619 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003dd0 con 0x5627ee04e580 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f58c00700 1 -- 172.21.15.86:0/3510845619 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f54099f90 con 0x5627ee04e580 2024-03-22T19:19:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f58c00700 1 -- 172.21.15.86:0/3510845619 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54097d50 con 0x5627ee04e580 2024-03-22T19:19:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3510845619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04e580 msgr2=0x5627ee04ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04e580 0x5627ee04ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f54099820 tx=0x7f9f54007b90 comp rx=0 tx=0).stop 2024-03-22T19:19:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3510845619 shutdown_connections 2024-03-22T19:19:29.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee04f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04dc20 0x5627ee04e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3510845619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04e580 0x5627ee04ee40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3510845619 >> 172.21.15.86:0/3510845619 conn(0x5627edfc3220 msgr2=0x5627ee048bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3510845619 shutdown_connections 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.693+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3510845619 wait complete. 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 -- start start 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 Processor -- start 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee057ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee058000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627edf3bd80 con 0x5627ee04e580 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627edf3ba20 con 0x5627ee04dc20 2024-03-22T19:19:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.694+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ede236a0 con 0x5627ee04f420 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f7de8cb80 10 client.0 _ll_register_callbacks cb 0x5627edda73a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee057ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee058000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee057ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39934/0 (socket says 172.21.15.86:39934) 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 -- 172.21.15.86:0/1739155368 learned_addr learned my addr 172.21.15.86:0/1739155368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63e00700 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 -- 172.21.15.86:0/1739155368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 msgr2=0x5627ee05f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 -- 172.21.15.86:0/1739155368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 msgr2=0x5627ee058000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee058000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 -- 172.21.15.86:0/1739155368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641a1460 con 0x5627ee04dc20 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f63400700 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee057ac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9f54096300 tx=0x7f9f54000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b762c9768fbceb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f4d600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:19:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.695+0000 7f9f4d600700 1 -- 172.21.15.86:0/1739155368 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5409d6c0 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f4d600700 1 -- 172.21.15.86:0/1739155368 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9f540b9300 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627ee040ad0 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f4d600700 1 -- 172.21.15.86:0/1739155368 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f54007c00 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 10 client.26411 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5627eddcdb60 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f4d600700 1 -- 172.21.15.86:0/1739155368 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627ee040ad0 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f4d600700 1 -- 172.21.15.86:0/1739155368 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9f54098d80 con 0x5627ee04dc20 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 10 client.26411 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 10 client.26411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 20 client.26411 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee063410 0x5627ee065830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 4 client.26411 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.696+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627edf98700 con 0x5627ee063410 2024-03-22T19:19:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.697+0000 7f9f62a00700 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee063410 0x5627ee065830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.697+0000 7f9f62a00700 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee063410 0x5627ee065830 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e704c63a3b6578 server_cookie=5da58848c69826c2 in_seq=0 out_seq=0 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.697+0000 7f9f4d600700 10 client.26411 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f4d600700 1 -- 172.21.15.86:0/1739155368 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5627edf98700 con 0x5627ee063410 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f4d600700 10 client.26411 handle_command_reply: tid=0 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 client.26411 shutdown 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 msgr2=0x5627ee057ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee057ac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9f54096300 tx=0x7f9f54000bc0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 shutdown_connections 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee063410 0x5627ee065830 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee057ac0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/1739155368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee058000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f4d600700 0 client.26411 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 >> 172.21.15.86:0/1739155368 conn(0x5627edfc3220 msgr2=0x5627ee0481f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 shutdown_connections 2024-03-22T19:19:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.698+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/1739155368 wait complete. 2024-03-22T19:19:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.699+0000 7f9f7de8cb80 20 client.26411 trim_cache size 0 max 16384 2024-03-22T19:19:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.715+0000 7f9f7de8cb80 1 Processor -- start 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.715+0000 7f9f7de8cb80 1 -- start start 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04dc20 0x5627ee05cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee056fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04f420 0x5627ee04f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ee013940 con 0x5627ee04e580 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627edf98700 con 0x5627ee04f420 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ee043210 con 0x5627ee04dc20 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee056fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee056fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37092/0 (socket says 172.21.15.86:37092) 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 -- 172.21.15.86:0/3545988676 learned_addr learned my addr 172.21.15.86:0/3545988676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f63e00700 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04f420 0x5627ee04f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f63400700 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04dc20 0x5627ee05cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 -- 172.21.15.86:0/3545988676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04dc20 msgr2=0x5627ee05cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04dc20 0x5627ee05cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 -- 172.21.15.86:0/3545988676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04f420 msgr2=0x5627ee04f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04f420 0x5627ee04f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.716+0000 7f9f62a00700 1 -- 172.21.15.86:0/3545988676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627edf65ea0 con 0x5627ee04e580 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.717+0000 7f9f62a00700 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee056fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000fb70 tx=0x7f9f50005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686e8b4ec142bab server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.717+0000 7f9f59600700 1 -- 172.21.15.86:0/3545988676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50028160 con 0x5627ee04e580 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.717+0000 7f9f59600700 1 -- 172.21.15.86:0/3545988676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5001d040 con 0x5627ee04e580 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.717+0000 7f9f59600700 1 -- 172.21.15.86:0/3545988676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000baf0 con 0x5627ee04e580 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.717+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3545988676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 msgr2=0x5627ee056fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.717+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee056fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000fb70 tx=0x7f9f50005dc0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3545988676 shutdown_connections 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04f420 0x5627ee04f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04dc20 0x5627ee05cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/3545988676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee056fd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3545988676 >> 172.21.15.86:0/3545988676 conn(0x5627edfc3220 msgr2=0x5627ee0567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3545988676 shutdown_connections 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/3545988676 wait complete. 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 -- start start 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.718+0000 7f9f7de8cb80 1 Processor -- start 2024-03-22T19:19:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee05ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee05cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ee05b9b0 con 0x5627ee04e580 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ee05baf0 con 0x5627ee04dc20 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ee05bcb0 con 0x5627ee04f420 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f7de8cb80 10 client.0 _ll_register_callbacks cb 0x7f9f64075140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee05cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f63400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee05ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f63400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee05ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:39964/0 (socket says 172.21.15.86:39964) 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f63400700 1 -- 172.21.15.86:0/2065139296 learned_addr learned my addr 172.21.15.86:0/2065139296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f63e00700 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f62a00700 1 -- 172.21.15.86:0/2065139296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 msgr2=0x5627ee05b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f62a00700 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f62a00700 1 -- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 msgr2=0x5627ee05ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f62a00700 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee05ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.719+0000 7f9f62a00700 1 -- 172.21.15.86:0/2065139296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f64109ad0 con 0x5627ee04e580 2024-03-22T19:19:29.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:29 smithi131 ceph-mon[78559]: pgmap v541: 97 pgs: 97 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 227 op/s 2024-03-22T19:19:29.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f62a00700 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee05cfd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000fa40 tx=0x7f9f5000f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f392e1a895e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f4d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:29.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f4d600700 1 -- 172.21.15.86:0/2065139296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50028360 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f54099f90 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f4d600700 1 -- 172.21.15.86:0/2065139296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f9f5000b890 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f4d600700 1 -- 172.21.15.86:0/2065139296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f5000ba90 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f7de8cb80 10 client.16287 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5627ede25cd0 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f4d600700 1 -- 172.21.15.86:0/2065139296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f50042020 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.720+0000 7f9f4d600700 1 -- 172.21.15.86:0/2065139296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9f50009950 con 0x5627ee04e580 2024-03-22T19:19:29.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f7de8cb80 10 client.16287 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f7de8cb80 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:19:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f7de8cb80 20 client.16287 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee0b5f20 0x5627ee0b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f7de8cb80 4 client.16287 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2024-03-22T19:19:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5627eddeecf0 con 0x5627ee0b5f20 2024-03-22T19:19:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f63400700 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee0b5f20 0x5627ee0b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f63400700 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee0b5f20 0x5627ee0b8310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f384bd9f53ad3f server_cookie=6d4272e69602e272 in_seq=0 out_seq=0 2024-03-22T19:19:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.721+0000 7f9f4d600700 10 client.16287 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f4d600700 1 -- 172.21.15.86:0/2065139296 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5627eddeecf0 con 0x5627ee0b5f20 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f4d600700 10 client.16287 handle_command_reply: tid=0 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 client.16287 shutdown 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 msgr2=0x5627ee05cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee05cfd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000fa40 tx=0x7f9f5000f480 comp rx=0 tx=0).stop 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 shutdown_connections 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5627ee0b5f20 0x5627ee0b8310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627ee04f420 0x5627ee05b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627ee04dc20 0x5627ee05ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 --2- 172.21.15.86:0/2065139296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ee04e580 0x5627ee05cfd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f4d600700 0 client.16287 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 >> 172.21.15.86:0/2065139296 conn(0x5627edfc3220 msgr2=0x5627ee055910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 shutdown_connections 2024-03-22T19:19:29.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 1 -- 172.21.15.86:0/2065139296 wait complete. 2024-03-22T19:19:29.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.722+0000 7f9f7de8cb80 20 client.16287 trim_cache size 0 max 16384 2024-03-22T19:19:29.731 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:19:29.731 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T19:19:29.731 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T19:19:29.731 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f480a0a90 msgr2=0x7f9f480a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f480a0a90 0x7f9f480a2f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009fd0 tx=0x7f9f5c00b040 comp rx=0 tx=0).stop 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 msgr2=0x7f9f64074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64074ac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e970 tx=0x7f9f5400ec80 comp rx=0 tx=0).stop 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 shutdown_connections 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f9f480a0a90 0x7f9f480a2f50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641028d0 0x7f9f640736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f64100d40 0x7f9f64074ac0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 --2- 172.21.15.86:0/2165783740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f64101f30 0x7f9f640731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 >> 172.21.15.86:0/2165783740 conn(0x7f9f640fc2b0 msgr2=0x7f9f64105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 shutdown_connections 2024-03-22T19:19:29.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.723+0000 7f9f6a400700 1 -- 172.21.15.86:0/2165783740 wait complete. 2024-03-22T19:19:29.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:29 smithi086 ceph-mon[38032]: pgmap v541: 97 pgs: 97 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 227 op/s 2024-03-22T19:19:29.741 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T19:19:29.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:19:29.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:29 smithi167 ceph-mon[47728]: pgmap v541: 97 pgs: 97 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 227 op/s 2024-03-22T19:19:29.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 -- 172.21.15.86:0/2467414920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 msgr2=0x7f0d5c109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 --2- 172.21.15.86:0/2467414920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c109090 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400b0d0 tx=0x7f0d5400b3e0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 -- 172.21.15.86:0/2467414920 shutdown_connections 2024-03-22T19:19:29.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 --2- 172.21.15.86:0/2467414920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c109090 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 --2- 172.21.15.86:0/2467414920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d5c100c50 0x7f0d5c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 --2- 172.21.15.86:0/2467414920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c101e40 0x7f0d5c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 -- 172.21.15.86:0/2467414920 >> 172.21.15.86:0/2467414920 conn(0x7f0d5c0fc1c0 msgr2=0x7f0d5c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:29.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 -- 172.21.15.86:0/2467414920 shutdown_connections 2024-03-22T19:19:29.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.933+0000 7f0d64400700 1 -- 172.21.15.86:0/2467414920 wait complete. 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.934+0000 7f0d64400700 1 Processor -- start 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.934+0000 7f0d64400700 1 -- start start 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d64400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c100c50 0x7f0d5c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d64400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d5c101e40 0x7f0d5c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d64400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d64400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1a0a70 con 0x7f0d5c1027e0 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d64400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1a0bb0 con 0x7f0d5c101e40 2024-03-22T19:19:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d64400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1a0d40 con 0x7f0d5c100c50 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c1a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37122/0 (socket says 172.21.15.86:37122) 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 -- 172.21.15.86:0/79365907 learned_addr learned my addr 172.21.15.86:0/79365907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d61000700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d5c101e40 0x7f0d5c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d61a00700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c100c50 0x7f0d5c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 -- 172.21.15.86:0/79365907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c100c50 msgr2=0x7f0d5c198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c100c50 0x7f0d5c198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 -- 172.21.15.86:0/79365907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d5c101e40 msgr2=0x7f0d5c1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d5c101e40 0x7f0d5c1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 -- 172.21.15.86:0/79365907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54009d00 con 0x7f0d5c1027e0 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.935+0000 7f0d62400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c1a04a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d54007b60 tx=0x7f0d54008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.936+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400e040 con 0x7f0d5c1027e0 2024-03-22T19:19:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.936+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d5401d930 con 0x7f0d5c1027e0 2024-03-22T19:19:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.936+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c1a0fc0 con 0x7f0d5c1027e0 2024-03-22T19:19:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.936+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5401b910 con 0x7f0d5c1027e0 2024-03-22T19:19:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.936+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c1a1390 con 0x7f0d5c1027e0 2024-03-22T19:19:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.937+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c0679e0 con 0x7f0d5c1027e0 2024-03-22T19:19:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.939+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0d5401daa0 con 0x7f0d5c1027e0 2024-03-22T19:19:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.940+0000 7f0d5a000700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d440a0a90 0x7f0d440a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:29.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.940+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f0d540cd6c0 con 0x7f0d5c1027e0 2024-03-22T19:19:29.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.940+0000 7f0d61a00700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d440a0a90 0x7f0d440a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:29.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.941+0000 7f0d61a00700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d440a0a90 0x7f0d440a2f50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c101ca0 tx=0x7f0d50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:29.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:29.944+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0d54097110 con 0x7f0d5c1027e0 2024-03-22T19:19:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.199+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d5c1a1520 con 0x7f0d5c1027e0 2024-03-22T19:19:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.199+0000 7f0d5a000700 1 -- 172.21.15.86:0/79365907 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10931 (secure 0 0 0) 0x7f0d5409a730 con 0x7f0d5c1027e0 2024-03-22T19:19:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:19:30.200 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:19:16.508755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:19:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d440a0a90 msgr2=0x7f0d440a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d440a0a90 0x7f0d440a2f50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c101ca0 tx=0x7f0d50009260 comp rx=0 tx=0).stop 2024-03-22T19:19:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 msgr2=0x7f0d5c1a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c1a04a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d54007b60 tx=0x7f0d54008e40 comp rx=0 tx=0).stop 2024-03-22T19:19:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 shutdown_connections 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0d440a0a90 0x7f0d440a2f50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c1027e0 0x7f0d5c1a04a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c100c50 0x7f0d5c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 --2- 172.21.15.86:0/79365907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d5c101e40 0x7f0d5c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 >> 172.21.15.86:0/79365907 conn(0x7f0d5c0fc1c0 msgr2=0x7f0d5c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 shutdown_connections 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.202+0000 7f0d64400700 1 -- 172.21.15.86:0/79365907 wait complete. 2024-03-22T19:19:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-03-22T19:19:30.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.418+0000 7fe4bcc00700 1 -- 172.21.15.86:0/3709845872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b81017d0 msgr2=0x7fe4b8103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.418+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/3709845872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b81017d0 0x7fe4b8103bc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009990 tx=0x7fe4a8009ca0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 -- 172.21.15.86:0/3709845872 shutdown_connections 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/3709845872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b8106a30 0x7fe4b8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/3709845872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b81017d0 0x7fe4b8103bc0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/3709845872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b8104100 0x7fe4b81064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 -- 172.21.15.86:0/3709845872 >> 172.21.15.86:0/3709845872 conn(0x7fe4b80fb1c0 msgr2=0x7fe4b80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 -- 172.21.15.86:0/3709845872 shutdown_connections 2024-03-22T19:19:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.419+0000 7fe4bcc00700 1 -- 172.21.15.86:0/3709845872 wait complete. 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 Processor -- start 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 -- start start 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b81017d0 0x7fe4b8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 0x7fe4b8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b8106a30 0x7fe4b819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b819e7f0 con 0x7fe4b8104100 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b819e960 con 0x7fe4b8106a30 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4bcc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b819eb20 con 0x7fe4b81017d0 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.420+0000 7fe4afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 0x7fe4b8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b81017d0 0x7fe4b8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 0x7fe4b8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37134/0 (socket says 172.21.15.86:37134) 2024-03-22T19:19:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4afe00700 1 -- 172.21.15.86:0/2924081216 learned_addr learned my addr 172.21.15.86:0/2924081216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b6800700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b8106a30 0x7fe4b819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 -- 172.21.15.86:0/2924081216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b8106a30 msgr2=0x7fe4b819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b8106a30 0x7fe4b819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 -- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 msgr2=0x7fe4b8197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 0x7fe4b8197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 -- 172.21.15.86:0/2924081216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80095f0 con 0x7fe4b81017d0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4afe00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 0x7fe4b8197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b81017d0 0x7fe4b8196cd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000c00 tx=0x7fe4a8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.421+0000 7fe4aea00700 1 -- 172.21.15.86:0/2924081216 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a801d070 con 0x7fe4b81017d0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.422+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b819eda0 con 0x7fe4b81017d0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.422+0000 7fe4aea00700 1 -- 172.21.15.86:0/2924081216 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4a8021410 con 0x7fe4b81017d0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.422+0000 7fe4aea00700 1 -- 172.21.15.86:0/2924081216 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8017650 con 0x7fe4b81017d0 2024-03-22T19:19:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.422+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b819f1d0 con 0x7fe4b81017d0 2024-03-22T19:19:30.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.425+0000 7fe4aea00700 1 -- 172.21.15.86:0/2924081216 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe4a8021580 con 0x7fe4b81017d0 2024-03-22T19:19:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.426+0000 7fe4d07c1b80 1 Processor -- start 2024-03-22T19:19:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.426+0000 7fe4d07c1b80 1 -- start start 2024-03-22T19:19:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200d9c20 0x5600200da040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200da580 0x5600200dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020032990 con 0x5600200db420 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020030350 con 0x5600200d9c20 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56002002bc40 con 0x5600200da580 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4b6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200d9c20 0x5600200da040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4b5e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200da580 0x5600200dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4b6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200d9c20 0x5600200da040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40016/0 (socket says 172.21.15.86:40016) 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4b6800700 1 -- 172.21.15.86:0/1472667519 learned_addr learned my addr 172.21.15.86:0/1472667519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4afe00700 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4aea00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe49c0a0a90 0x7fe49c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4aea00700 1 -- 172.21.15.86:0/2924081216 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe4a80cd2b0 con 0x7fe4b81017d0 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4b6800700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe49c0a0a90 0x7fe49c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4afe00700 1 -- 172.21.15.86:0/1472667519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200da580 msgr2=0x5600200dae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.427+0000 7fe4afe00700 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200da580 0x5600200dae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4afe00700 1 -- 172.21.15.86:0/1472667519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200d9c20 msgr2=0x5600200da040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4afe00700 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200d9c20 0x5600200da040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4afe00700 1 -- 172.21.15.86:0/1472667519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001fead3b0 con 0x5600200db420 2024-03-22T19:19:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4afe00700 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200db8a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0005950 tx=0x7fe4a000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05b2ccf224ca984 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4a7000700 1 -- 172.21.15.86:0/1472667519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0011640 con 0x5600200db420 2024-03-22T19:19:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4b6800700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe49c0a0a90 0x7fe49c0a2f50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000fb80 tx=0x7fe4b0004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4a7000700 1 -- 172.21.15.86:0/1472667519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4a0015070 con 0x5600200db420 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.428+0000 7fe4a7000700 1 -- 172.21.15.86:0/1472667519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0019ae0 con 0x5600200db420 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/1472667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 msgr2=0x5600200db8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200db8a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0005950 tx=0x7fe4a000bf90 comp rx=0 tx=0).stop 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/1472667519 shutdown_connections 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200db8a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200d9c20 0x5600200da040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/1472667519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200da580 0x5600200dae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/1472667519 >> 172.21.15.86:0/1472667519 conn(0x56002004f220 msgr2=0x5600200d4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/1472667519 shutdown_connections 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.429+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/1472667519 wait complete. 2024-03-22T19:19:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.430+0000 7fe4d07c1b80 1 -- start start 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.430+0000 7fe4d07c1b80 1 Processor -- start 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.430+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200d9c20 0x5600200e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001ffc7d80 con 0x5600200db420 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001ffc7a20 con 0x5600200da580 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001feaf6a0 con 0x5600200d9c20 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4d07c1b80 10 client.0 _ll_register_callbacks cb 0x56001fe333a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40024/0 (socket says 172.21.15.86:40024) 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4afe00700 1 -- 172.21.15.86:0/3721062318 learned_addr learned my addr 172.21.15.86:0/3721062318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200d9c20 0x5600200e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b6800700 1 -- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200d9c20 msgr2=0x5600200e39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b6800700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200d9c20 0x5600200e39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b6800700 1 -- 172.21.15.86:0/3721062318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 msgr2=0x5600200e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b6800700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4b6800700 1 -- 172.21.15.86:0/3721062318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a000be00 con 0x5600200db420 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.431+0000 7fe4afe00700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.432+0000 7fe4b6800700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200eaf60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0018640 tx=0x7fe4b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc66026728c4358 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.432+0000 7fe49b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.433+0000 7fe49b400700 1 -- 172.21.15.86:0/3721062318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000e040 con 0x5600200db420 2024-03-22T19:19:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.433+0000 7fe49b400700 1 -- 172.21.15.86:0/3721062318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe4b0003e10 con 0x5600200db420 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.433+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600200ccad0 con 0x5600200db420 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe49b400700 1 -- 172.21.15.86:0/3721062318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4b0007990 con 0x5600200db420 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 10 client.16305 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56001fe59b60 con 0x5600200db420 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe49b400700 1 -- 172.21.15.86:0/3721062318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600200ccad0 con 0x5600200db420 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe49b400700 1 -- 172.21.15.86:0/3721062318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe4b002e070 con 0x5600200db420 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 10 client.16305 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 20 client.16305 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5600200ef8d0 0x5600200f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 4 client.16305 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.434+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560020024700 con 0x5600200ef8d0 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.435+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5600200ef8d0 0x5600200f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.435+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5600200ef8d0 0x5600200f1cf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64c2b74b44160c47 server_cookie=1833af5df1ae25a6 in_seq=0 out_seq=0 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.435+0000 7fe49b400700 10 client.16305 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe49b400700 1 -- 172.21.15.86:0/3721062318 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560020024700 con 0x5600200ef8d0 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe49b400700 10 client.16305 handle_command_reply: tid=0 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 client.16305 shutdown 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 msgr2=0x5600200eaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200eaf60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0018640 tx=0x7fe4b0004ad0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 shutdown_connections 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200db420 0x5600200eaf60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x5600200ef8d0 0x5600200f1cf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200d9c20 0x5600200e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe49b400700 0 client.16305 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/3721062318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.436+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 >> 172.21.15.86:0/3721062318 conn(0x56002004f220 msgr2=0x5600200d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.437+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 shutdown_connections 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.437+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/3721062318 wait complete. 2024-03-22T19:19:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.437+0000 7fe4d07c1b80 20 client.16305 trim_cache size 0 max 16384 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 Processor -- start 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 -- start start 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600200eccc0 con 0x5600200d9c20 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600200cc610 con 0x5600200da580 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.450+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600200e4660 con 0x5600200db420 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37158/0 (socket says 172.21.15.86:37158) 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b5e00700 1 -- 172.21.15.86:0/2457008989 learned_addr learned my addr 172.21.15.86:0/2457008989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42510/0 (socket says 172.21.15.86:42510) 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4afe00700 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 -- 172.21.15.86:0/2457008989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 msgr2=0x5600200db820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200db820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 -- 172.21.15.86:0/2457008989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 msgr2=0x5600200e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 -- 172.21.15.86:0/2457008989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001fff1ea0 con 0x5600200db420 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4b6800700 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e2ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0004830 tx=0x7fe4b0018fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d5b126d56ff686 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.451+0000 7fe4a7a00700 1 -- 172.21.15.86:0/2457008989 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000e030 con 0x5600200db420 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4a7a00700 1 -- 172.21.15.86:0/2457008989 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4b000f040 con 0x5600200db420 2024-03-22T19:19:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4a7a00700 1 -- 172.21.15.86:0/2457008989 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b001aca0 con 0x5600200db420 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/2457008989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 msgr2=0x5600200e2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e2ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0004830 tx=0x7fe4b0018fd0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/2457008989 shutdown_connections 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e2ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/2457008989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/2457008989 >> 172.21.15.86:0/2457008989 conn(0x56002004f220 msgr2=0x5600200d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/2457008989 shutdown_connections 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.452+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/2457008989 wait complete. 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 -- start start 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 Processor -- start 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600200e83f0 con 0x5600200d9c20 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600200e8560 con 0x5600200da580 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.453+0000 7fe4d07c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600200e8720 con 0x5600200db420 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4d07c1b80 10 client.0 _ll_register_callbacks cb 0x7fe4b00093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4b6800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40034/0 (socket says 172.21.15.86:40034) 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 -- 172.21.15.86:0/506389364 learned_addr learned my addr 172.21.15.86:0/506389364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 -- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 msgr2=0x5600200e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 -- 172.21.15.86:0/506389364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 msgr2=0x5600200e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 -- 172.21.15.86:0/506389364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b819f1d0 con 0x5600200da580 2024-03-22T19:19:30.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe4afe00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e75f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0011010 tx=0x7fe4a00050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3cca04aaf30413f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.454+0000 7fe49b400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe49b400700 1 -- 172.21.15.86:0/506389364 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00054f0 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4a0015070 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe49b400700 1 -- 172.21.15.86:0/506389364 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe4a0005650 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe49b400700 1 -- 172.21.15.86:0/506389364 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4a0019790 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 10 client.26441 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56001fe59b60 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe49b400700 1 -- 172.21.15.86:0/506389364 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4a0034020 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe49b400700 1 -- 172.21.15.86:0/506389364 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe4a002c060 con 0x5600200da580 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 10 client.26441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-22T19:19:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 20 client.26441 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.455+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560020142070 0x560020144490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe4d07c1b80 4 client.26441 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:19:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 --> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56001fe7acf0 con 0x560020142070 2024-03-22T19:19:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560020142070 0x560020144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe49b400700 10 client.26441 ms_handle_connect on v2:172.21.15.167:6834/2161889252 2024-03-22T19:19:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe4b5e00700 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560020142070 0x560020144490 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d75e37c077c70c8c server_cookie=1c7a2333bb5227e5 in_seq=0 out_seq=0 2024-03-22T19:19:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe49b400700 1 -- 172.21.15.86:0/506389364 <== mds.4 v2:172.21.15.167:6834/2161889252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56001fe7acf0 con 0x560020142070 2024-03-22T19:19:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe49b400700 10 client.26441 handle_command_reply: tid=0 2024-03-22T19:19:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.456+0000 7fe4d07c1b80 1 client.26441 shutdown 2024-03-22T19:19:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 msgr2=0x5600200e75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e75f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0011010 tx=0x7fe4a00050d0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 shutdown_connections 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:6834/2161889252,v1:172.21.15.167:6835/2161889252] conn(0x560020142070 0x560020144490 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600200db420 0x5600200e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600200d9c20 0x5600200e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe49b400700 0 client.26441 ms_handle_reset on v2:172.21.15.167:6834/2161889252 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 --2- 172.21.15.86:0/506389364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600200da580 0x5600200e75f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 >> 172.21.15.86:0/506389364 conn(0x56002004f220 msgr2=0x5600200e2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 shutdown_connections 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 1 -- 172.21.15.86:0/506389364 wait complete. 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4d07c1b80 20 client.26441 trim_cache size 0 max 16384 2024-03-22T19:19:30.464 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe49c0a0a90 msgr2=0x7fe49c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.457+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe49c0a0a90 0x7fe49c0a2f50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000fb80 tx=0x7fe4b0004b40 comp rx=0 tx=0).stop 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b81017d0 msgr2=0x7fe4b8196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b81017d0 0x7fe4b8196cd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000c00 tx=0x7fe4a8005740 comp rx=0 tx=0).stop 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 shutdown_connections 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe49c0a0a90 0x7fe49c0a2f50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b8106a30 0x7fe4b819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b81017d0 0x7fe4b8196cd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 --2- 172.21.15.86:0/2924081216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b8104100 0x7fe4b8197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 >> 172.21.15.86:0/2924081216 conn(0x7fe4b80fb1c0 msgr2=0x7fe4b81036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 shutdown_connections 2024-03-22T19:19:30.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.458+0000 7fe4bcc00700 1 -- 172.21.15.86:0/2924081216 wait complete. 2024-03-22T19:19:30.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:30 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/79365907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:19:30.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 -- 172.21.15.86:0/3656484704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c101d80 msgr2=0x7f7e2c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 --2- 172.21.15.86:0/3656484704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c101d80 0x7f7e2c1021e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e180099c0 tx=0x7f7e18009cd0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 -- 172.21.15.86:0/3656484704 shutdown_connections 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 --2- 172.21.15.86:0/3656484704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c102720 0x7f7e2c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 --2- 172.21.15.86:0/3656484704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e2c100b90 0x7f7e2c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 --2- 172.21.15.86:0/3656484704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c101d80 0x7f7e2c1021e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 -- 172.21.15.86:0/3656484704 >> 172.21.15.86:0/3656484704 conn(0x7f7e2c0fc0e0 msgr2=0x7f7e2c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 -- 172.21.15.86:0/3656484704 shutdown_connections 2024-03-22T19:19:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.670+0000 7f7e32600700 1 -- 172.21.15.86:0/3656484704 wait complete. 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 Processor -- start 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 -- start start 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 0x7f7e2c196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e2c101d80 0x7f7e2c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c102720 0x7f7e2c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c19e660 con 0x7f7e2c100b90 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c19e7a0 con 0x7f7e2c102720 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.671+0000 7f7e32600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c19e930 con 0x7f7e2c101d80 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 0x7f7e2c196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 0x7f7e2c196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37184/0 (socket says 172.21.15.86:37184) 2024-03-22T19:19:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31200700 1 -- 172.21.15.86:0/3436649727 learned_addr learned my addr 172.21.15.86:0/3436649727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c102720 0x7f7e2c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e2be00700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e2c101d80 0x7f7e2c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 -- 172.21.15.86:0/3436649727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e2c101d80 msgr2=0x7f7e2c197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e2c101d80 0x7f7e2c197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 -- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 msgr2=0x7f7e2c196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 0x7f7e2c196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 -- 172.21.15.86:0/3436649727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e180095f0 con 0x7f7e2c102720 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31200700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 0x7f7e2c196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.672+0000 7f7e31c00700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c102720 0x7f7e2c19e090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400efc0 tx=0x7f7e2400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.673+0000 7f7e29600700 1 -- 172.21.15.86:0/3436649727 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24018070 con 0x7f7e2c102720 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.673+0000 7f7e29600700 1 -- 172.21.15.86:0/3436649727 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e2400f040 con 0x7f7e2c102720 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.673+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c19ebb0 con 0x7f7e2c102720 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.673+0000 7f7e29600700 1 -- 172.21.15.86:0/3436649727 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400cd40 con 0x7f7e2c102720 2024-03-22T19:19:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.673+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c19f0c0 con 0x7f7e2c102720 2024-03-22T19:19:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.676+0000 7f7e29600700 1 -- 172.21.15.86:0/3436649727 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f7e24003a00 con 0x7f7e2c102720 2024-03-22T19:19:30.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.677+0000 7f7e29600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e140a0a90 0x7f7e140a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.677+0000 7f7e4780db80 1 Processor -- start 2024-03-22T19:19:30.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.677+0000 7f7e29600700 1 -- 172.21.15.86:0/3436649727 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7e240cbbd0 con 0x7f7e2c102720 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.677+0000 7f7e4780db80 1 -- start start 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.677+0000 7f7e31200700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e140a0a90 0x7f7e140a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84638dc0 0x55de846391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de8463a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8463a5e0 0x55de8463aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e4780db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de84594990 con 0x55de84638dc0 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e4780db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de84592350 con 0x55de8463a5e0 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e4780db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8458dc40 con 0x55de84639720 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84638dc0 0x55de846391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84638dc0 0x55de846391e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37200/0 (socket says 172.21.15.86:37200) 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8463a5e0 0x55de8463aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31200700 1 -- 172.21.15.86:0/1113512342 learned_addr learned my addr 172.21.15.86:0/1113512342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e2be00700 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de8463a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31200700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e140a0a90 0x7f7e140a2f50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005950 tx=0x7f7e20011950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31c00700 1 -- 172.21.15.86:0/1113512342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 msgr2=0x55de8463a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31c00700 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de8463a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31c00700 1 -- 172.21.15.86:0/1113512342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84638dc0 msgr2=0x55de846391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31c00700 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84638dc0 0x55de846391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.678+0000 7f7e31c00700 1 -- 172.21.15.86:0/1113512342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de8440f3b0 con 0x55de8463a5e0 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e31c00700 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8463a5e0 0x55de8463aa60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400ef90 tx=0x7f7e240990b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b02eb5dad6b082 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e1e000700 1 -- 172.21.15.86:0/1113512342 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24098c40 con 0x55de8463a5e0 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e1e000700 1 -- 172.21.15.86:0/1113512342 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e24004ad0 con 0x55de8463a5e0 2024-03-22T19:19:30.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e1e000700 1 -- 172.21.15.86:0/1113512342 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24097760 con 0x55de8463a5e0 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 -- 172.21.15.86:0/1113512342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8463a5e0 msgr2=0x55de8463aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8463a5e0 0x55de8463aa60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400ef90 tx=0x7f7e240990b0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 -- 172.21.15.86:0/1113512342 shutdown_connections 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de8463a5e0 0x55de8463aa60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84638dc0 0x55de846391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1113512342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de8463a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.679+0000 7f7e4780db80 1 -- 172.21.15.86:0/1113512342 >> 172.21.15.86:0/1113512342 conn(0x55de845b1220 msgr2=0x55de84636c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.680+0000 7f7e4780db80 1 -- 172.21.15.86:0/1113512342 shutdown_connections 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.680+0000 7f7e4780db80 1 -- 172.21.15.86:0/1113512342 wait complete. 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.680+0000 7f7e4780db80 1 -- start start 2024-03-22T19:19:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.680+0000 7f7e4780db80 1 Processor -- start 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de8464e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 0x55de8464e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8463a5e0 0x55de84655910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de84529d80 con 0x55de84639720 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de84529a20 con 0x55de84638dc0 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de844116a0 con 0x55de8463a5e0 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e4780db80 10 client.0 _ll_register_callbacks cb 0x55de843953a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 0x55de8464e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 0x55de8464e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37206/0 (socket says 172.21.15.86:37206) 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e31200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de8464e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 -- 172.21.15.86:0/1023897522 learned_addr learned my addr 172.21.15.86:0/1023897522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e31c00700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8463a5e0 0x55de84655910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 -- 172.21.15.86:0/1023897522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8463a5e0 msgr2=0x55de84655910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8463a5e0 0x55de84655910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 -- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 msgr2=0x55de8464e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de8464e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 -- 172.21.15.86:0/1023897522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c19f0c0 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e31c00700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8463a5e0 0x55de84655910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.681+0000 7f7e2be00700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 0x55de8464e8d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e180095c0 tx=0x7f7e18005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc75fe6d7b75a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e12a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e12a00700 1 -- 172.21.15.86:0/1023897522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18003e50 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e12a00700 1 -- 172.21.15.86:0/1023897522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7e18005040 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de8462ec30 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e12a00700 1 -- 172.21.15.86:0/1023897522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e18005200 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e4780db80 10 client.16323 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55de843bbb60 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e12a00700 1 -- 172.21.15.86:0/1023897522 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e18033080 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e12a00700 1 -- 172.21.15.86:0/1023897522 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7e18024d20 con 0x55de84639720 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e4780db80 10 client.16323 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e4780db80 10 client.16323 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.682+0000 7f7e4780db80 20 client.16323 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.683+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de8465a1c0 0x55de8465c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.683+0000 7f7e4780db80 4 client.16323 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.683+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de84586700 con 0x55de8465a1c0 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.683+0000 7f7e31200700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de8465a1c0 0x55de8465c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.683+0000 7f7e12a00700 10 client.16323 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.683+0000 7f7e31200700 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de8465a1c0 0x55de8465c5e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce2a67be4b5bdc88 server_cookie=cac82f830e31255c in_seq=0 out_seq=0 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.684+0000 7f7e12a00700 1 -- 172.21.15.86:0/1023897522 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55de84586700 con 0x55de8465a1c0 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.684+0000 7f7e12a00700 10 client.16323 handle_command_reply: tid=0 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.684+0000 7f7e4780db80 1 client.16323 shutdown 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 msgr2=0x55de8464e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 0x55de8464e8d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e180095c0 tx=0x7f7e18005e80 comp rx=0 tx=0).stop 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 shutdown_connections 2024-03-22T19:19:30.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de8465a1c0 0x55de8465c5e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de8463a5e0 0x55de84655910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e12a00700 0 client.16323 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de8464e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 --2- 172.21.15.86:0/1023897522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de84639720 0x55de8464e8d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 >> 172.21.15.86:0/1023897522 conn(0x55de845b1220 msgr2=0x55de84636be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 shutdown_connections 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 1 -- 172.21.15.86:0/1023897522 wait complete. 2024-03-22T19:19:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.685+0000 7f7e4780db80 20 client.16323 trim_cache size 0 max 16384 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 Processor -- start 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 -- start start 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84638dc0 0x55de846558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84639720 0x55de8463f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de8463a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8464c560 con 0x55de8463a5e0 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de84649e50 con 0x55de84639720 2024-03-22T19:19:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.698+0000 7f7e4780db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de84637a30 con 0x55de84638dc0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de8463a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de8463a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37222/0 (socket says 172.21.15.86:37222) 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 -- 172.21.15.86:0/424336749 learned_addr learned my addr 172.21.15.86:0/424336749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31200700 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84638dc0 0x55de846558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e2be00700 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84639720 0x55de8463f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 -- 172.21.15.86:0/424336749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84638dc0 msgr2=0x55de846558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84638dc0 0x55de846558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 -- 172.21.15.86:0/424336749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84639720 msgr2=0x55de8463f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84639720 0x55de8463f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 -- 172.21.15.86:0/424336749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de84553ea0 con 0x55de8463a5e0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e2be00700 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84639720 0x55de8463f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e31c00700 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de8463a9e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e24098fe0 tx=0x7f7e24004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f178f3a0b3e1b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.699+0000 7f7e1ea00700 1 -- 172.21.15.86:0/424336749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2409d110 con 0x55de8463a5e0 2024-03-22T19:19:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e1ea00700 1 -- 172.21.15.86:0/424336749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e240999e0 con 0x55de8463a5e0 2024-03-22T19:19:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e1ea00700 1 -- 172.21.15.86:0/424336749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2401b560 con 0x55de8463a5e0 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 -- 172.21.15.86:0/424336749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 msgr2=0x55de8463a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de8463a9e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e24098fe0 tx=0x7f7e24004d20 comp rx=0 tx=0).stop 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 -- 172.21.15.86:0/424336749 shutdown_connections 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de8463a9e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84638dc0 0x55de846558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 --2- 172.21.15.86:0/424336749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84639720 0x55de8463f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 -- 172.21.15.86:0/424336749 >> 172.21.15.86:0/424336749 conn(0x55de845b1220 msgr2=0x55de84632b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 -- 172.21.15.86:0/424336749 shutdown_connections 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.700+0000 7f7e4780db80 1 -- 172.21.15.86:0/424336749 wait complete. 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 -- start start 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 Processor -- start 2024-03-22T19:19:30.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de846a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de846a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de846aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de846ab120 con 0x55de8463a5e0 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de846ab290 con 0x55de84638dc0 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de846ab450 con 0x55de84639720 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.701+0000 7f7e4780db80 10 client.0 _ll_register_callbacks cb 0x7f7e1800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de846aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de846a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de846a83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42578/0 (socket says 172.21.15.86:42578) 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e2be00700 1 -- 172.21.15.86:0/2559226326 learned_addr learned my addr 172.21.15.86:0/2559226326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31200700 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de846a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 -- 172.21.15.86:0/2559226326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 msgr2=0x55de846a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de846a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 -- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 msgr2=0x55de846a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de846a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 -- 172.21.15.86:0/2559226326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de843bbb60 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e31c00700 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de846aa9e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7e24095d50 tx=0x7f7e24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be69ed513f395ad server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e12a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e12a00700 1 -- 172.21.15.86:0/2559226326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240992e0 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e12a00700 1 -- 172.21.15.86:0/2559226326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7e2409d110 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.702+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e24004ad0 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e12a00700 1 -- 172.21.15.86:0/2559226326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e24099540 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 10 client.16329 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55de8440f3b0 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e12a00700 1 -- 172.21.15.86:0/2559226326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e24004ad0 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e12a00700 1 -- 172.21.15.86:0/2559226326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7e240b28b0 con 0x55de8463a5e0 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 10 client.16329 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:30.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 10 client.16329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 20 client.16329 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de846ac9a0 0x55de846aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 4 client.16329 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 --> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de843dccf0 con 0x55de846ac9a0 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.703+0000 7f7e31200700 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de846ac9a0 0x55de846aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e12a00700 10 client.16329 ms_handle_connect on v2:172.21.15.131:6836/3746834114 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e31200700 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de846ac9a0 0x55de846aed90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c2750dd01bf37e2 server_cookie=7710c9f48234408d in_seq=0 out_seq=0 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e12a00700 1 -- 172.21.15.86:0/2559226326 <== mds.3 v2:172.21.15.131:6836/3746834114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de843dccf0 con 0x55de846ac9a0 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e12a00700 10 client.16329 handle_command_reply: tid=0 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 client.16329 shutdown 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 msgr2=0x55de846aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de846aa9e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7e24095d50 tx=0x7f7e24009f90 comp rx=0 tx=0).stop 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 shutdown_connections 2024-03-22T19:19:30.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8463a5e0 0x55de846aa9e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de84638dc0 0x55de846a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.131:6836/3746834114,v1:172.21.15.131:6837/3746834114] conn(0x55de846ac9a0 0x55de846aed90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 --2- 172.21.15.86:0/2559226326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de84639720 0x55de846a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 >> 172.21.15.86:0/2559226326 conn(0x55de845b1220 msgr2=0x55de84632b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e12a00700 0 client.16329 ms_handle_reset on v2:172.21.15.131:6836/3746834114 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.704+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 shutdown_connections 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e4780db80 1 -- 172.21.15.86:0/2559226326 wait complete. 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e4780db80 20 client.16329 trim_cache size 0 max 16384 2024-03-22T19:19:30.710 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:19:30.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e140a0a90 msgr2=0x7f7e140a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e140a0a90 0x7f7e140a2f50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005950 tx=0x7f7e20011950 comp rx=0 tx=0).stop 2024-03-22T19:19:30.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c102720 msgr2=0x7f7e2c19e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c102720 0x7f7e2c19e090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400efc0 tx=0x7f7e2400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 shutdown_connections 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e140a0a90 0x7f7e140a2f50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c102720 0x7f7e2c19e090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c100b90 0x7f7e2c196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 --2- 172.21.15.86:0/3436649727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e2c101d80 0x7f7e2c197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 >> 172.21.15.86:0/3436649727 conn(0x7f7e2c0fc0e0 msgr2=0x7f7e2c105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 shutdown_connections 2024-03-22T19:19:30.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.705+0000 7f7e32600700 1 -- 172.21.15.86:0/3436649727 wait complete. 2024-03-22T19:19:30.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:30 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/79365907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:19:30.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-22T19:19:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:30 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/79365907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 -- 172.21.15.86:0/1836278046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c108c10 msgr2=0x7fab1c10b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 --2- 172.21.15.86:0/1836278046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c108c10 0x7fab1c10b080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fab14009f90 tx=0x7fab1400b330 comp rx=0 tx=0).stop 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 -- 172.21.15.86:0/1836278046 shutdown_connections 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 --2- 172.21.15.86:0/1836278046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c108c10 0x7fab1c10b080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 --2- 172.21.15.86:0/1836278046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c074d90 0x7fab1c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 --2- 172.21.15.86:0/1836278046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c0737b0 0x7fab1c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 -- 172.21.15.86:0/1836278046 >> 172.21.15.86:0/1836278046 conn(0x7fab1c0fc2f0 msgr2=0x7fab1c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 -- 172.21.15.86:0/1836278046 shutdown_connections 2024-03-22T19:19:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.918+0000 7fab21400700 1 -- 172.21.15.86:0/1836278046 wait complete. 2024-03-22T19:19:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 Processor -- start 2024-03-22T19:19:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 -- start start 2024-03-22T19:19:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c0737b0 0x7fab1c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c074d90 0x7fab1c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 0x7fab1c1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a0aa0 con 0x7fab1c108c10 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a0be0 con 0x7fab1c074d90 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.919+0000 7fab21400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a0d70 con 0x7fab1c0737b0 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 0x7fab1c1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1a800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c0737b0 0x7fab1c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 0x7fab1c1a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37244/0 (socket says 172.21.15.86:37244) 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 -- 172.21.15.86:0/2311305300 learned_addr learned my addr 172.21.15.86:0/2311305300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab19e00700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c074d90 0x7fab1c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 -- 172.21.15.86:0/2311305300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c0737b0 msgr2=0x7fab1c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c0737b0 0x7fab1c198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 -- 172.21.15.86:0/2311305300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c074d90 msgr2=0x7fab1c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c074d90 0x7fab1c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 -- 172.21.15.86:0/2311305300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080095f0 con 0x7fab1c108c10 2024-03-22T19:19:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1a800700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c0737b0 0x7fab1c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab1b200700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 0x7fab1c1a04d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b6c0 tx=0x7fab14004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.920+0000 7fab13400700 1 -- 172.21.15.86:0/2311305300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400e040 con 0x7fab1c108c10 2024-03-22T19:19:30.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.921+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab14009c40 con 0x7fab1c108c10 2024-03-22T19:19:30.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.921+0000 7fab13400700 1 -- 172.21.15.86:0/2311305300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1400bbf0 con 0x7fab1c108c10 2024-03-22T19:19:30.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.921+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a1230 con 0x7fab1c108c10 2024-03-22T19:19:30.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.921+0000 7fab13400700 1 -- 172.21.15.86:0/2311305300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14005040 con 0x7fab1c108c10 2024-03-22T19:19:30.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.924+0000 7fab13400700 1 -- 172.21.15.86:0/2311305300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fab1401d470 con 0x7fab1c108c10 2024-03-22T19:19:30.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.925+0000 7fab34e73b80 1 Processor -- start 2024-03-22T19:19:30.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 -- start start 2024-03-22T19:19:30.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab13400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0a90 0x7fab000a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c44040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3c44e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab13400700 1 -- 172.21.15.86:0/2311305300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fab140cd220 con 0x7fab1c108c10 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3b9c990 con 0x5610e3c44580 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3b9a350 con 0x5610e3c45420 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab34e73b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3b95c40 con 0x5610e3c43c20 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3c44e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3c44e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37254/0 (socket says 172.21.15.86:37254) 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab19e00700 1 -- 172.21.15.86:0/3874467446 learned_addr learned my addr 172.21.15.86:0/3874467446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab1a800700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c44040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.926+0000 7fab19e00700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0a90 0x7fab000a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab1b200700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 -- 172.21.15.86:0/3874467446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 msgr2=0x5610e3c44040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c44040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 -- 172.21.15.86:0/3874467446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 msgr2=0x5610e3c458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 -- 172.21.15.86:0/3874467446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610e3a173b0 con 0x5610e3c44580 2024-03-22T19:19:30.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab1b200700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab1a800700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c44040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3c44e40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fab08011e00 tx=0x7fab0800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e30f6e1e87db602 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.927+0000 7fab19e00700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0a90 0x7fab000a2f50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fab0800f9f0 tx=0x7fab08018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab07400700 1 -- 172.21.15.86:0/3874467446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08011050 con 0x5610e3c44580 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab07400700 1 -- 172.21.15.86:0/3874467446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab08009d50 con 0x5610e3c44580 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab07400700 1 -- 172.21.15.86:0/3874467446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800ec10 con 0x5610e3c44580 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 -- 172.21.15.86:0/3874467446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 msgr2=0x5610e3c44e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3c44e40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fab08011e00 tx=0x7fab0800f6a0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 -- 172.21.15.86:0/3874467446 shutdown_connections 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c44040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3874467446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3c44e40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 -- 172.21.15.86:0/3874467446 >> 172.21.15.86:0/3874467446 conn(0x5610e3bb9220 msgr2=0x5610e3c3ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.928+0000 7fab34e73b80 1 -- 172.21.15.86:0/3874467446 shutdown_connections 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.929+0000 7fab34e73b80 1 -- 172.21.15.86:0/3874467446 wait complete. 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.929+0000 7fab34e73b80 1 -- start start 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.929+0000 7fab34e73b80 1 Processor -- start 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 0x5610e3c4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c44580 0x5610e3c4e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c55050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3b31d80 con 0x5610e3c43c20 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3b31a20 con 0x5610e3c45420 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3a196a0 con 0x5610e3c44580 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab34e73b80 10 client.0 _ll_register_callbacks cb 0x5610e399d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab1a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 0x5610e3c4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab1a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 0x5610e3c4dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37268/0 (socket says 172.21.15.86:37268) 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab1a800700 1 -- 172.21.15.86:0/3821044182 learned_addr learned my addr 172.21.15.86:0/3821044182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab19e00700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c44580 0x5610e3c4e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.930+0000 7fab1b200700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c55050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1a800700 1 -- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c44580 msgr2=0x5610e3c4e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1a800700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c44580 0x5610e3c4e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1a800700 1 -- 172.21.15.86:0/3821044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 msgr2=0x5610e3c55050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1a800700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c55050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1a800700 1 -- 172.21.15.86:0/3821044182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a1230 con 0x5610e3c43c20 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1b200700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c55050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab1a800700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 0x5610e3c4dad0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fab0c012d20 tx=0x7fab0c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2c7b6f3e062aab server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7faafbe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7faafbe00700 1 -- 172.21.15.86:0/3821044182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c017070 con 0x5610e3c43c20 2024-03-22T19:19:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7faafbe00700 1 -- 172.21.15.86:0/3821044182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fab0c00fbb0 con 0x5610e3c43c20 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610e3c36ad0 con 0x5610e3c43c20 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7faafbe00700 1 -- 172.21.15.86:0/3821044182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0c00fd70 con 0x5610e3c43c20 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab34e73b80 10 client.16341 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.931+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fab08019040 con 0x5610e3c43c20 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7faafbe00700 1 -- 172.21.15.86:0/3821044182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610e3c36ad0 con 0x5610e3c43c20 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7faafbe00700 1 -- 172.21.15.86:0/3821044182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab0c005460 con 0x5610e3c43c20 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab34e73b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab34e73b80 10 client.16341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab34e73b80 20 client.16341 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3c598c0 0x5610e3c5bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab34e73b80 4 client.16341 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610e3b8e700 con 0x5610e3c598c0 2024-03-22T19:19:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.932+0000 7fab19e00700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3c598c0 0x5610e3c5bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.933+0000 7fab19e00700 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3c598c0 0x5610e3c5bce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afed5e93520592c9 server_cookie=64264ab10b4bd923 in_seq=0 out_seq=0 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.933+0000 7faafbe00700 10 client.16341 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7faafbe00700 1 -- 172.21.15.86:0/3821044182 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5610e3b8e700 con 0x5610e3c598c0 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7faafbe00700 10 client.16341 handle_command_reply: tid=0 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 client.16341 shutdown 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 msgr2=0x5610e3c4dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 0x5610e3c4dad0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fab0c012d20 tx=0x7fab0c00f6a0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 shutdown_connections 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3c598c0 0x5610e3c5bce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3c55050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c43c20 0x5610e3c4dad0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7faafbe00700 0 client.16341 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:19:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 --2- 172.21.15.86:0/3821044182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c44580 0x5610e3c4e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 >> 172.21.15.86:0/3821044182 conn(0x5610e3bb9220 msgr2=0x5610e3c3e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 shutdown_connections 2024-03-22T19:19:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 1 -- 172.21.15.86:0/3821044182 wait complete. 2024-03-22T19:19:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.934+0000 7fab34e73b80 20 client.16341 trim_cache size 0 max 16384 2024-03-22T19:19:30.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.947+0000 7fab34e73b80 1 Processor -- start 2024-03-22T19:19:30.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.947+0000 7fab34e73b80 1 -- start start 2024-03-22T19:19:30.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.947+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c60640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.947+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c44580 0x5610e3c53000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.947+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c45420 0x5610e3c45800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab34e73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3c4c4d0 con 0x5610e3c45420 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab34e73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3c4b610 con 0x5610e3c44580 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab34e73b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3c48f20 con 0x5610e3c43c20 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1a800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c60640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c45420 0x5610e3c45800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1a800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c60640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42638/0 (socket says 172.21.15.86:42638) 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1a800700 1 -- 172.21.15.86:0/1991763929 learned_addr learned my addr 172.21.15.86:0/1991763929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab19e00700 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c44580 0x5610e3c53000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 -- 172.21.15.86:0/1991763929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 msgr2=0x5610e3c60640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c60640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 -- 172.21.15.86:0/1991763929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c44580 msgr2=0x5610e3c53000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c44580 0x5610e3c53000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 -- 172.21.15.86:0/1991763929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610e39c3b60 con 0x5610e3c45420 2024-03-22T19:19:30.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1a800700 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c60640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:30.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.948+0000 7fab1b200700 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c45420 0x5610e3c45800 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fab14009790 tx=0x7fab140cabe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4980358a5a76ab87 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab07e00700 1 -- 172.21.15.86:0/1991763929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140cc070 con 0x5610e3c45420 2024-03-22T19:19:30.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab07e00700 1 -- 172.21.15.86:0/1991763929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab140b8e90 con 0x5610e3c45420 2024-03-22T19:19:30.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab07e00700 1 -- 172.21.15.86:0/1991763929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140a2d20 con 0x5610e3c45420 2024-03-22T19:19:30.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 -- 172.21.15.86:0/1991763929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c45420 msgr2=0x5610e3c45800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c45420 0x5610e3c45800 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fab14009790 tx=0x7fab140cabe0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 -- 172.21.15.86:0/1991763929 shutdown_connections 2024-03-22T19:19:30.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c45420 0x5610e3c45800 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3c60640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1991763929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c44580 0x5610e3c53000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 -- 172.21.15.86:0/1991763929 >> 172.21.15.86:0/1991763929 conn(0x5610e3bb9220 msgr2=0x5610e3c4b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 -- 172.21.15.86:0/1991763929 shutdown_connections 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.949+0000 7fab34e73b80 1 -- 172.21.15.86:0/1991763929 wait complete. 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 -- start start 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 Processor -- start 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3ca35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3ca3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3caa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3caab60 con 0x5610e3c44580 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3caaca0 con 0x5610e3c45420 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e3caae30 con 0x5610e3c43c20 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab34e73b80 10 client.0 _ll_register_callbacks cb 0x7fab0c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.950+0000 7fab19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3ca3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3ca3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37278/0 (socket says 172.21.15.86:37278) 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 -- 172.21.15.86:0/1227923926 learned_addr learned my addr 172.21.15.86:0/1227923926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab1a800700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3ca35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab1b200700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3caa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 -- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 msgr2=0x5610e3ca35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3ca35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 -- 172.21.15.86:0/1227923926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 msgr2=0x5610e3caa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3caa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 -- 172.21.15.86:0/1227923926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08019040 con 0x5610e3c44580 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab1b200700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3caa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab19e00700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3ca3b20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fab08010010 tx=0x7fab08009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9170ab0eb21f524 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7faafbe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7faafbe00700 1 -- 172.21.15.86:0/1227923926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08036070 con 0x5610e3c44580 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7faafbe00700 1 -- 172.21.15.86:0/1227923926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fab080050b0 con 0x5610e3c44580 2024-03-22T19:19:30.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.951+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab08009d50 con 0x5610e3c44580 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7faafbe00700 1 -- 172.21.15.86:0/1227923926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab080119e0 con 0x5610e3c44580 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 10 client.16347 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fab14009c40 con 0x5610e3c44580 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7faafbe00700 1 -- 172.21.15.86:0/1227923926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab08040020 con 0x5610e3c44580 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7faafbe00700 1 -- 172.21.15.86:0/1227923926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab08039030 con 0x5610e3c44580 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 20 client.16347 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3cac7f0 0x5610e3caec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 4 client.16347 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 --> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610e39e4cf0 con 0x5610e3cac7f0 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.952+0000 7fab1a800700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3cac7f0 0x5610e3caec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:30.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab1a800700 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3cac7f0 0x5610e3caec10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ff1c79ff50bfd43 server_cookie=e0a2da23c4b6f151 in_seq=0 out_seq=0 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7faafbe00700 10 client.16347 ms_handle_connect on v2:172.21.15.167:6838/3747705607 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7faafbe00700 1 -- 172.21.15.86:0/1227923926 <== mds.2 v2:172.21.15.167:6838/3747705607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610e39e4cf0 con 0x5610e3cac7f0 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7faafbe00700 10 client.16347 handle_command_reply: tid=0 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 client.16347 shutdown 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 msgr2=0x5610e3ca3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3ca3b20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fab08010010 tx=0x7fab08009ec0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 shutdown_connections 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:6838/3747705607,v1:172.21.15.167:6839/3747705607] conn(0x5610e3cac7f0 0x5610e3caec10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610e3c45420 0x5610e3caa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7faafbe00700 0 client.16347 ms_handle_reset on v2:172.21.15.167:6838/3747705607 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610e3c43c20 0x5610e3ca35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 --2- 172.21.15.86:0/1227923926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610e3c44580 0x5610e3ca3b20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 >> 172.21.15.86:0/1227923926 conn(0x5610e3bb9220 msgr2=0x5610e3c483b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 shutdown_connections 2024-03-22T19:19:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.953+0000 7fab34e73b80 1 -- 172.21.15.86:0/1227923926 wait complete. 2024-03-22T19:19:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab34e73b80 20 client.16347 trim_cache size 0 max 16384 2024-03-22T19:19:30.960 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0a90 msgr2=0x7fab000a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0a90 0x7fab000a2f50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fab0800f9f0 tx=0x7fab08018000 comp rx=0 tx=0).stop 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 msgr2=0x7fab1c1a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 0x7fab1c1a04d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b6c0 tx=0x7fab14004ce0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 shutdown_connections 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fab000a0a90 0x7fab000a2f50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c108c10 0x7fab1c1a04d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab1c0737b0 0x7fab1c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 --2- 172.21.15.86:0/2311305300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c074d90 0x7fab1c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 >> 172.21.15.86:0/2311305300 conn(0x7fab1c0fc2f0 msgr2=0x7fab1c104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 shutdown_connections 2024-03-22T19:19:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:30.954+0000 7fab21400700 1 -- 172.21.15.86:0/2311305300 wait complete. 2024-03-22T19:19:30.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-22T19:19:31.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 -- 172.21.15.86:0/438011484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3050104100 msgr2=0x7f30501064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 --2- 172.21.15.86:0/438011484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3050104100 0x7f30501064f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f303c009990 tx=0x7f303c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 -- 172.21.15.86:0/438011484 shutdown_connections 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 --2- 172.21.15.86:0/438011484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3050106a30 0x7f3050108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 --2- 172.21.15.86:0/438011484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30501017d0 0x7f3050103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 --2- 172.21.15.86:0/438011484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3050104100 0x7f30501064f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 -- 172.21.15.86:0/438011484 >> 172.21.15.86:0/438011484 conn(0x7f30500fb1c0 msgr2=0x7f30500fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.165+0000 7f3056600700 1 -- 172.21.15.86:0/438011484 shutdown_connections 2024-03-22T19:19:31.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.166+0000 7f3056600700 1 -- 172.21.15.86:0/438011484 wait complete. 2024-03-22T19:19:31.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.166+0000 7f3056600700 1 Processor -- start 2024-03-22T19:19:31.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 -- start start 2024-03-22T19:19:31.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 0x7f3050196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3050104100 0x7f3050197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3050106a30 0x7f305019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305019e800 con 0x7f30501017d0 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305019e970 con 0x7f3050106a30 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f3056600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305019eb30 con 0x7f3050104100 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f304f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 0x7f3050196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f304f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 0x7f3050196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37288/0 (socket says 172.21.15.86:37288) 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f304f400700 1 -- 172.21.15.86:0/2108257710 learned_addr learned my addr 172.21.15.86:0/2108257710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f304ea00700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3050104100 0x7f3050197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.167+0000 7f304fe00700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3050106a30 0x7f305019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f304f400700 1 -- 172.21.15.86:0/2108257710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3050104100 msgr2=0x7f3050197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f304f400700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3050104100 0x7f3050197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f304f400700 1 -- 172.21.15.86:0/2108257710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3050106a30 msgr2=0x7f305019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f304f400700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3050106a30 0x7f305019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f304f400700 1 -- 172.21.15.86:0/2108257710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0095f0 con 0x7f30501017d0 2024-03-22T19:19:31.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f304f400700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 0x7f3050196ce0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f304000d950 tx=0x7f304000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f3047e00700 1 -- 172.21.15.86:0/2108257710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304000f890 con 0x7f30501017d0 2024-03-22T19:19:31.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305019ee10 con 0x7f30501017d0 2024-03-22T19:19:31.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f3047e00700 1 -- 172.21.15.86:0/2108257710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3040010460 con 0x7f30501017d0 2024-03-22T19:19:31.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f3047e00700 1 -- 172.21.15.86:0/2108257710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304000e610 con 0x7f30501017d0 2024-03-22T19:19:31.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.168+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305019f240 con 0x7f30501017d0 2024-03-22T19:19:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.172+0000 7f3047e00700 1 -- 172.21.15.86:0/2108257710 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f30400108d0 con 0x7f30501017d0 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.173+0000 7f306a10fb80 1 Processor -- start 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 -- start start 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f3047e00700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f30340a0a90 0x7f30340a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca5c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577eca5dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ec9b0990 con 0x5577eca5c8d0 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ec9ae350 con 0x5577eca5d800 2024-03-22T19:19:31.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f306a10fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ec9a9c40 con 0x5577eca5bf70 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f3047e00700 1 -- 172.21.15.86:0/2108257710 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f30400cbd50 con 0x7f30501017d0 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f304f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca5c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f304f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca5c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42668/0 (socket says 172.21.15.86:42668) 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f304f400700 1 -- 172.21.15.86:0/1797350917 learned_addr learned my addr 172.21.15.86:0/1797350917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f304fe00700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f30340a0a90 0x7f30340a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f304ea00700 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.174+0000 7f304fe00700 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577eca5dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304f400700 1 -- 172.21.15.86:0/1797350917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 msgr2=0x5577eca5dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304f400700 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577eca5dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304f400700 1 -- 172.21.15.86:0/1797350917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 msgr2=0x5577eca5d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304f400700 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca5d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304f400700 1 -- 172.21.15.86:0/1797350917 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ec82b3b0 con 0x5577eca5bf70 2024-03-22T19:19:31.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304ea00700 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca5d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304f400700 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca5c390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3040005950 tx=0x7f3040099030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab23117e6a2dc91 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f304fe00700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f30340a0a90 0x7f30340a2f50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f30480078d0 tx=0x7f304800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f3044c00700 1 -- 172.21.15.86:0/1797350917 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040048c20 con 0x5577eca5bf70 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f3044c00700 1 -- 172.21.15.86:0/1797350917 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3040099b70 con 0x5577eca5bf70 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.175+0000 7f3044c00700 1 -- 172.21.15.86:0/1797350917 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400996d0 con 0x5577eca5bf70 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 -- 172.21.15.86:0/1797350917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 msgr2=0x5577eca5c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca5c390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3040005950 tx=0x7f3040099030 comp rx=0 tx=0).stop 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 -- 172.21.15.86:0/1797350917 shutdown_connections 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577eca5dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca5c390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1797350917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca5d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 -- 172.21.15.86:0/1797350917 >> 172.21.15.86:0/1797350917 conn(0x5577ec9cd220 msgr2=0x5577eca529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 -- 172.21.15.86:0/1797350917 shutdown_connections 2024-03-22T19:19:31.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.176+0000 7f306a10fb80 1 -- 172.21.15.86:0/1797350917 wait complete. 2024-03-22T19:19:31.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 -- start start 2024-03-22T19:19:31.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 Processor -- start 2024-03-22T19:19:31.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5bf70 0x5577eca65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca66420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5d800 0x5577eca6d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ec945d80 con 0x5577eca5c8d0 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ec945a20 con 0x5577eca5bf70 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ec82d6a0 con 0x5577eca5d800 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.177+0000 7f306a10fb80 10 client.0 _ll_register_callbacks cb 0x5577ec7b13a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca66420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5bf70 0x5577eca65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca66420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37316/0 (socket says 172.21.15.86:37316) 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304ea00700 1 -- 172.21.15.86:0/1010407076 learned_addr learned my addr 172.21.15.86:0/1010407076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304fe00700 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5d800 0x5577eca6d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 -- 172.21.15.86:0/1010407076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5d800 msgr2=0x5577eca6d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5d800 0x5577eca6d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 -- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 msgr2=0x5577eca66420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca66420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 -- 172.21.15.86:0/1010407076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305019f240 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f304f400700 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5bf70 0x5577eca65ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f304000b2f0 tx=0x7f304004b130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcb0dd2b8c56aab server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.178+0000 7f3039600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f3039600700 1 -- 172.21.15.86:0/1010407076 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040049190 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f3039600700 1 -- 172.21.15.86:0/1010407076 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f3040048c20 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577eca4aad0 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f3039600700 1 -- 172.21.15.86:0/1010407076 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30400a18f0 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f306a10fb80 10 client.26489 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5577ec7d7b60 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f3039600700 1 -- 172.21.15.86:0/1010407076 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577eca4aad0 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f3039600700 1 -- 172.21.15.86:0/1010407076 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f30400a1460 con 0x5577eca5bf70 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f306a10fb80 10 client.26489 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f306a10fb80 10 client.26489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.179+0000 7f306a10fb80 20 client.26489 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.180+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577eca71880 0x5577eca73ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.180+0000 7f306a10fb80 4 client.26489 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.180+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577ec9a2700 con 0x5577eca71880 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.180+0000 7f304ea00700 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577eca71880 0x5577eca73ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.180+0000 7f304ea00700 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577eca71880 0x5577eca73ca0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cf5182b4a38a506 server_cookie=a7cfc3e4534f053a in_seq=0 out_seq=0 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.180+0000 7f3039600700 10 client.26489 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f3039600700 1 -- 172.21.15.86:0/1010407076 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5577ec9a2700 con 0x5577eca71880 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f3039600700 10 client.26489 handle_command_reply: tid=0 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 client.26489 shutdown 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5bf70 msgr2=0x5577eca65ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5bf70 0x5577eca65ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f304000b2f0 tx=0x7f304004b130 comp rx=0 tx=0).stop 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 shutdown_connections 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577eca71880 0x5577eca73ca0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5d800 0x5577eca6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5bf70 0x5577eca65ee0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f3039600700 0 client.26489 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:19:31.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1010407076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5c8d0 0x5577eca66420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 >> 172.21.15.86:0/1010407076 conn(0x5577ec9cd220 msgr2=0x5577eca5ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 shutdown_connections 2024-03-22T19:19:31.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.181+0000 7f306a10fb80 1 -- 172.21.15.86:0/1010407076 wait complete. 2024-03-22T19:19:31.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.182+0000 7f306a10fb80 20 client.26489 trim_cache size 0 max 16384 2024-03-22T19:19:31.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 Processor -- start 2024-03-22T19:19:31.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 -- start start 2024-03-22T19:19:31.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca6b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 0x5577eca5ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5d800 0x5577eca64bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577eca5bb70 con 0x5577eca5d800 2024-03-22T19:19:31.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577eca63980 con 0x5577eca5c8d0 2024-03-22T19:19:31.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f306a10fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577eca61270 con 0x5577eca5bf70 2024-03-22T19:19:31.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f304fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5d800 0x5577eca64bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f304ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 0x5577eca5ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f304ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 0x5577eca5ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:40198/0 (socket says 172.21.15.86:40198) 2024-03-22T19:19:31.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f304ea00700 1 -- 172.21.15.86:0/1450602648 learned_addr learned my addr 172.21.15.86:0/1450602648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.195+0000 7f304f400700 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca6b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f304ea00700 1 -- 172.21.15.86:0/1450602648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 msgr2=0x5577eca6b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f304ea00700 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca6b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f304ea00700 1 -- 172.21.15.86:0/1450602648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5d800 msgr2=0x5577eca64bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f304ea00700 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5d800 0x5577eca64bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f304ea00700 1 -- 172.21.15.86:0/1450602648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ec96fea0 con 0x5577eca5c8d0 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f304ea00700 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 0x5577eca5ccb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f303c0098e0 tx=0x7f303c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757d3a328afc6adb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f3045600700 1 -- 172.21.15.86:0/1450602648 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c01d040 con 0x5577eca5c8d0 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f3045600700 1 -- 172.21.15.86:0/1450602648 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303c01e040 con 0x5577eca5c8d0 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.196+0000 7f3045600700 1 -- 172.21.15.86:0/1450602648 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c003af0 con 0x5577eca5c8d0 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 -- 172.21.15.86:0/1450602648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 msgr2=0x5577eca5ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 0x5577eca5ccb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f303c0098e0 tx=0x7f303c005fd0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 -- 172.21.15.86:0/1450602648 shutdown_connections 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5d800 0x5577eca64bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5bf70 0x5577eca6b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 --2- 172.21.15.86:0/1450602648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5c8d0 0x5577eca5ccb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 -- 172.21.15.86:0/1450602648 >> 172.21.15.86:0/1450602648 conn(0x5577ec9cd220 msgr2=0x5577eca4ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 -- 172.21.15.86:0/1450602648 shutdown_connections 2024-03-22T19:19:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 -- 172.21.15.86:0/1450602648 wait complete. 2024-03-22T19:19:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.197+0000 7f306a10fb80 1 -- start start 2024-03-22T19:19:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 Processor -- start 2024-03-22T19:19:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 0x5577ecabb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5c8d0 0x5577ecabbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577ecac2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ecac2ad0 con 0x5577eca5bf70 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ecac2c10 con 0x5577eca5d800 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ecac2dd0 con 0x5577eca5c8d0 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f306a10fb80 10 client.0 _ll_register_callbacks cb 0x7f30400992f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f304f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 0x5577ecabb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f304ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5c8d0 0x5577ecabbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f304f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 0x5577ecabb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37338/0 (socket says 172.21.15.86:37338) 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f304f400700 1 -- 172.21.15.86:0/4096002567 learned_addr learned my addr 172.21.15.86:0/4096002567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.198+0000 7f304fe00700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577ecac2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304f400700 1 -- 172.21.15.86:0/4096002567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5c8d0 msgr2=0x5577ecabbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304f400700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5c8d0 0x5577ecabbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304f400700 1 -- 172.21.15.86:0/4096002567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 msgr2=0x5577ecac2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304f400700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577ecac2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304f400700 1 -- 172.21.15.86:0/4096002567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3050109a90 con 0x5577eca5bf70 2024-03-22T19:19:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304ea00700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5c8d0 0x5577ecabbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f304f400700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 0x5577ecabb5c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3040099400 tx=0x7f3040010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e82d41ee48416a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f3039600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f3039600700 1 -- 172.21.15.86:0/4096002567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040049190 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f3039600700 1 -- 172.21.15.86:0/4096002567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f30400a18f0 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3040099b70 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f3039600700 1 -- 172.21.15.86:0/4096002567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30400841d0 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f306a10fb80 10 client.16365 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5577ec82fcd0 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.199+0000 7f3039600700 1 -- 172.21.15.86:0/4096002567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3040099b70 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f3039600700 1 -- 172.21.15.86:0/4096002567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f30400983d0 con 0x5577eca5bf70 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f306a10fb80 10 client.16365 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f306a10fb80 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f306a10fb80 20 client.16365 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f306a10fb80 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577ecac44e0 0x5577ecac6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f306a10fb80 4 client.16365 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 --> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577ec7f8cf0 con 0x5577ecac44e0 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f304ea00700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577ecac44e0 0x5577ecac6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.200+0000 7f304ea00700 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577ecac44e0 0x5577ecac6900 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b05e4ace3b8bcba5 server_cookie=e0619035c902c842 in_seq=0 out_seq=0 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f3039600700 10 client.16365 ms_handle_connect on v2:172.21.15.86:6840/3747210669 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f3039600700 1 -- 172.21.15.86:0/4096002567 <== mds.1 v2:172.21.15.86:6840/3747210669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577ec7f8cf0 con 0x5577ecac44e0 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f3039600700 10 client.16365 handle_command_reply: tid=0 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 client.16365 shutdown 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 msgr2=0x5577ecabb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 0x5577ecabb5c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3040099400 tx=0x7f3040010cd0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 shutdown_connections 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:6840/3747210669,v1:172.21.15.86:6841/3747210669] conn(0x5577ecac44e0 0x5577ecac6900 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577eca5d800 0x5577ecac2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577eca5bf70 0x5577ecabb5c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 --2- 172.21.15.86:0/4096002567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577eca5c8d0 0x5577ecabbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f3039600700 0 client.16365 ms_handle_reset on v2:172.21.15.86:6840/3747210669 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 >> 172.21.15.86:0/4096002567 conn(0x5577ec9cd220 msgr2=0x5577eca64bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 shutdown_connections 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 1 -- 172.21.15.86:0/4096002567 wait complete. 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.201+0000 7f306a10fb80 20 client.16365 trim_cache size 0 max 16384 2024-03-22T19:19:31.207 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f30340a0a90 msgr2=0x7f30340a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f30340a0a90 0x7f30340a2f50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f30480078d0 tx=0x7f304800b040 comp rx=0 tx=0).stop 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 msgr2=0x7f3050196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 0x7f3050196ce0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f304000d950 tx=0x7f304000dc60 comp rx=0 tx=0).stop 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 shutdown_connections 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f30340a0a90 0x7f30340a2f50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3050106a30 0x7f305019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30501017d0 0x7f3050196ce0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 --2- 172.21.15.86:0/2108257710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3050104100 0x7f3050197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 >> 172.21.15.86:0/2108257710 conn(0x7f30500fb1c0 msgr2=0x7f30501036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 shutdown_connections 2024-03-22T19:19:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.202+0000 7f3056600700 1 -- 172.21.15.86:0/2108257710 wait complete. 2024-03-22T19:19:31.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 -- 172.21.15.86:0/544407068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac100d50 msgr2=0x7f50ac101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 --2- 172.21.15.86:0/544407068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac100d50 0x7f50ac101170 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099c0 tx=0x7f509c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 -- 172.21.15.86:0/544407068 shutdown_connections 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 --2- 172.21.15.86:0/544407068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac1028e0 0x7f50ac109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 --2- 172.21.15.86:0/544407068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac100d50 0x7f50ac101170 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 --2- 172.21.15.86:0/544407068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 0x7f50ac1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 -- 172.21.15.86:0/544407068 >> 172.21.15.86:0/544407068 conn(0x7f50ac0fc2c0 msgr2=0x7f50ac0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 -- 172.21.15.86:0/544407068 shutdown_connections 2024-03-22T19:19:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.414+0000 7f50b2600700 1 -- 172.21.15.86:0/544407068 wait complete. 2024-03-22T19:19:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.415+0000 7f50b2600700 1 Processor -- start 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.415+0000 7f50b2600700 1 -- start start 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50b2600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac100d50 0x7f50ac198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50b2600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 0x7f50ac199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50b2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 0x7f50ac1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50b2600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a0a60 con 0x7f50ac1028e0 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50b2600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a0ba0 con 0x7f50ac100d50 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50b2600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a0d30 con 0x7f50ac101f40 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 0x7f50ac1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 0x7f50ac1a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37356/0 (socket says 172.21.15.86:37356) 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50abe00700 1 -- 172.21.15.86:0/522626131 learned_addr learned my addr 172.21.15.86:0/522626131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50ab400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac100d50 0x7f50ac198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.416+0000 7f50aaa00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 0x7f50ac199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50abe00700 1 -- 172.21.15.86:0/522626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 msgr2=0x7f50ac199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50abe00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 0x7f50ac199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50abe00700 1 -- 172.21.15.86:0/522626131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac100d50 msgr2=0x7f50ac198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50abe00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac100d50 0x7f50ac198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50abe00700 1 -- 172.21.15.86:0/522626131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0095f0 con 0x7f50ac1028e0 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50aaa00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 0x7f50ac199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50abe00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 0x7f50ac1a0490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f50a400ef30 tx=0x7f50a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.417+0000 7f50a3e00700 1 -- 172.21.15.86:0/522626131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4018070 con 0x7f50ac1028e0 2024-03-22T19:19:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.418+0000 7f50a3e00700 1 -- 172.21.15.86:0/522626131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a400f040 con 0x7f50ac1028e0 2024-03-22T19:19:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.418+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac1a0fb0 con 0x7f50ac1028e0 2024-03-22T19:19:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.418+0000 7f50a3e00700 1 -- 172.21.15.86:0/522626131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400cdb0 con 0x7f50ac1028e0 2024-03-22T19:19:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.418+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a1430 con 0x7f50ac1028e0 2024-03-22T19:19:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.421+0000 7f50a3e00700 1 -- 172.21.15.86:0/522626131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f50a4003a00 con 0x7f50ac1028e0 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.422+0000 7f50c6114b80 1 Processor -- start 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.422+0000 7f50a3e00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50900a0a90 0x7f50900a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.422+0000 7f50c6114b80 1 -- start start 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.422+0000 7f50a3e00700 1 -- 172.21.15.86:0/522626131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f50a40cbaf0 con 0x7f50ac1028e0 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50aaa00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50900a0a90 0x7f50900a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 0x55883674f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x55883674fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558836750420 0x5588367508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50c6114b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588366a7990 con 0x55883674ec20 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50c6114b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588366a5350 con 0x558836750420 2024-03-22T19:19:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50c6114b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588366a0c40 con 0x55883674f580 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 0x55883674f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 0x55883674f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37358/0 (socket says 172.21.15.86:37358) 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 -- 172.21.15.86:0/4269483368 learned_addr learned my addr 172.21.15.86:0/4269483368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50abe00700 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558836750420 0x5588367508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50aaa00700 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x55883674fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 -- 172.21.15.86:0/4269483368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 msgr2=0x55883674fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x55883674fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 -- 172.21.15.86:0/4269483368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558836750420 msgr2=0x5588367508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558836750420 0x5588367508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50ab400700 1 -- 172.21.15.86:0/4269483368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588365223b0 con 0x55883674ec20 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50aaa00700 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x55883674fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.423+0000 7f50aaa00700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50900a0a90 0x7f50900a2f50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5098005fd0 tx=0x7f509800f640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.424+0000 7f50ab400700 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 0x55883674f040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f509c006010 tx=0x7f509c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af46ee41766f246a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.424+0000 7f50a0c00700 1 -- 172.21.15.86:0/4269483368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c00ba90 con 0x55883674ec20 2024-03-22T19:19:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.424+0000 7f50a0c00700 1 -- 172.21.15.86:0/4269483368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509c01e040 con 0x55883674ec20 2024-03-22T19:19:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.424+0000 7f50a0c00700 1 -- 172.21.15.86:0/4269483368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c003bb0 con 0x55883674ec20 2024-03-22T19:19:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 -- 172.21.15.86:0/4269483368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 msgr2=0x55883674f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 0x55883674f040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f509c006010 tx=0x7f509c005c70 comp rx=0 tx=0).stop 2024-03-22T19:19:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 -- 172.21.15.86:0/4269483368 shutdown_connections 2024-03-22T19:19:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558836750420 0x5588367508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674ec20 0x55883674f040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4269483368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x55883674fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 -- 172.21.15.86:0/4269483368 >> 172.21.15.86:0/4269483368 conn(0x5588366c4220 msgr2=0x558836749bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 -- 172.21.15.86:0/4269483368 shutdown_connections 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 -- 172.21.15.86:0/4269483368 wait complete. 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 -- start start 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.425+0000 7f50c6114b80 1 Processor -- start 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x558836758a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836758fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x55883675ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883663cd80 con 0x558836750420 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883663ca20 con 0x55883674ec20 2024-03-22T19:19:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588365246a0 con 0x55883674f580 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50c6114b80 10 client.0 _ll_register_callbacks cb 0x5588364a83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50aaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836758fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x55883675ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50aaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836758fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42734/0 (socket says 172.21.15.86:42734) 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50aaa00700 1 -- 172.21.15.86:0/3424674564 learned_addr learned my addr 172.21.15.86:0/3424674564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.426+0000 7f50ab400700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x558836758a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50aaa00700 1 -- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 msgr2=0x558836758a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50aaa00700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x558836758a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50aaa00700 1 -- 172.21.15.86:0/3424674564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 msgr2=0x55883675ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50aaa00700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x55883675ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50aaa00700 1 -- 172.21.15.86:0/3424674564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac109ae0 con 0x55883674f580 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50ab400700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x558836758a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50abe00700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x55883675ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:31.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50aaa00700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836758fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f50980124e0 tx=0x7f50980047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1550295c31252d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f5095600700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f5095600700 1 -- 172.21.15.86:0/3424674564 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509801d630 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558836741ad0 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.427+0000 7f5095600700 1 -- 172.21.15.86:0/3424674564 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5098018040 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f5095600700 1 -- 172.21.15.86:0/3424674564 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5098012e90 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 10 client.26494 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5588364ceb60 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f5095600700 1 -- 172.21.15.86:0/3424674564 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558836741ad0 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f5095600700 1 -- 172.21.15.86:0/3424674564 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f509802f070 con 0x55883674f580 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 10 client.26494 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 10 client.26494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 20 client.26494 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:31.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558836764990 0x558836766db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 4 client.26494 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.428+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558836699700 con 0x558836764990 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.429+0000 7f50ab400700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558836764990 0x558836766db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.429+0000 7f50ab400700 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558836764990 0x558836766db0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac533c4fe7ef1d14 server_cookie=59a5923823b09063 in_seq=0 out_seq=0 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.429+0000 7f5095600700 10 client.26494 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f5095600700 1 -- 172.21.15.86:0/3424674564 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558836699700 con 0x558836764990 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f5095600700 10 client.26494 handle_command_reply: tid=0 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 client.26494 shutdown 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 msgr2=0x558836758fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836758fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f50980124e0 tx=0x7f50980047c0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 shutdown_connections 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x558836764990 0x558836766db0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x55883675ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x558836758a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f5095600700 0 client.26494 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3424674564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836758fa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 >> 172.21.15.86:0/3424674564 conn(0x5588366c4220 msgr2=0x5588367491f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 shutdown_connections 2024-03-22T19:19:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 1 -- 172.21.15.86:0/3424674564 wait complete. 2024-03-22T19:19:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.430+0000 7f50c6114b80 20 client.26494 trim_cache size 0 max 16384 2024-03-22T19:19:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 Processor -- start 2024-03-22T19:19:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 -- start start 2024-03-22T19:19:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x55883675def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836750800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x558836760690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558836761d40 con 0x558836750420 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588367574d0 con 0x55883674ec20 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.444+0000 7f50c6114b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883674b3b0 con 0x55883674f580 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x558836760690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836750800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x558836760690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37368/0 (socket says 172.21.15.86:37368) 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50abe00700 1 -- 172.21.15.86:0/3792199361 learned_addr learned my addr 172.21.15.86:0/3792199361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50ab400700 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x55883675def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 -- 172.21.15.86:0/3792199361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 msgr2=0x55883675def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x55883675def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 -- 172.21.15.86:0/3792199361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 msgr2=0x558836760690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x558836760690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 -- 172.21.15.86:0/3792199361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558836666ea0 con 0x55883674f580 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50abe00700 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x558836760690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50aaa00700 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836750800 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f509801e9a0 tx=0x7f509800b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf74357aeef8cbd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50a1600700 1 -- 172.21.15.86:0/3792199361 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509801e430 con 0x55883674f580 2024-03-22T19:19:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50a1600700 1 -- 172.21.15.86:0/3792199361 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5098012e90 con 0x55883674f580 2024-03-22T19:19:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.445+0000 7f50a1600700 1 -- 172.21.15.86:0/3792199361 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098003be0 con 0x55883674f580 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 -- 172.21.15.86:0/3792199361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 msgr2=0x558836750800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836750800 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f509801e9a0 tx=0x7f509800b560 comp rx=0 tx=0).stop 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 -- 172.21.15.86:0/3792199361 shutdown_connections 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558836750420 0x558836760690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x55883675def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 --2- 172.21.15.86:0/3792199361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55883674f580 0x558836750800 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 -- 172.21.15.86:0/3792199361 >> 172.21.15.86:0/3792199361 conn(0x5588366c4220 msgr2=0x558836756910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 -- 172.21.15.86:0/3792199361 shutdown_connections 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.446+0000 7f50c6114b80 1 -- 172.21.15.86:0/3792199361 wait complete. 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 -- start start 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 Processor -- start 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x5588367b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 0x5588367b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558836750420 0x5588367b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588367b5450 con 0x55883674f580 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588367b5590 con 0x55883674ec20 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588367b5750 con 0x558836750420 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50c6114b80 10 client.0 _ll_register_callbacks cb 0x7f509800fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.447+0000 7f50aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 0x5588367b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50ab400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x5588367b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 0x5588367b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37378/0 (socket says 172.21.15.86:37378) 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 -- 172.21.15.86:0/4094218306 learned_addr learned my addr 172.21.15.86:0/4094218306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50abe00700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558836750420 0x5588367b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 -- 172.21.15.86:0/4094218306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558836750420 msgr2=0x5588367b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558836750420 0x5588367b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 -- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 msgr2=0x5588367b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x5588367b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 -- 172.21.15.86:0/4094218306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a1430 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50ab400700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x5588367b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50aaa00700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 0x5588367b29c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f509801ec10 tx=0x7f5098003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5383df0a1f41f327 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f5095600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f5095600700 1 -- 172.21.15.86:0/4094218306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509803f070 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f5095600700 1 -- 172.21.15.86:0/4094218306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f5098018040 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.448+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f509c01e040 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.449+0000 7f5095600700 1 -- 172.21.15.86:0/4094218306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f509801e7f0 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.449+0000 7f50c6114b80 10 client.16383 fetch_fsmap learned FSMap version 46 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.449+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5588364ceb60 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.449+0000 7f5095600700 1 -- 172.21.15.86:0/4094218306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509c01e040 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f5095600700 1 -- 172.21.15.86:0/4094218306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f509802f070 con 0x55883674f580 2024-03-22T19:19:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50c6114b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T19:19:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50c6114b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-22T19:19:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50c6114b80 20 client.16383 populate_metadata read hostname 'smithi086' 2024-03-22T19:19:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5588367b6dc0 0x5588367b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50c6114b80 4 client.16383 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2024-03-22T19:19:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 --> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588364efcf0 con 0x5588367b6dc0 2024-03-22T19:19:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.450+0000 7f50ab400700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5588367b6dc0 0x5588367b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f5095600700 10 client.16383 ms_handle_connect on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50ab400700 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5588367b6dc0 0x5588367b91e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97eb37b547826257 server_cookie=77269304ae63d0e9 in_seq=0 out_seq=0 2024-03-22T19:19:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f5095600700 1 -- 172.21.15.86:0/4094218306 <== mds.0 v2:172.21.15.131:6838/1545428523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588364efcf0 con 0x5588367b6dc0 2024-03-22T19:19:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f5095600700 10 client.16383 handle_command_reply: tid=0 2024-03-22T19:19:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 client.16383 shutdown 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 msgr2=0x5588367b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 0x5588367b29c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f509801ec10 tx=0x7f5098003b90 comp rx=0 tx=0).stop 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 shutdown_connections 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:6838/1545428523,v1:172.21.15.131:6839/1545428523] conn(0x5588367b6dc0 0x5588367b91e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558836750420 0x5588367b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55883674ec20 0x5588367b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 --2- 172.21.15.86:0/4094218306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883674f580 0x5588367b29c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f5095600700 0 client.16383 ms_handle_reset on v2:172.21.15.131:6838/1545428523 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 >> 172.21.15.86:0/4094218306 conn(0x5588366c4220 msgr2=0x558836757740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 shutdown_connections 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 1 -- 172.21.15.86:0/4094218306 wait complete. 2024-03-22T19:19:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.451+0000 7f50c6114b80 20 client.16383 trim_cache size 0 max 16384 2024-03-22T19:19:31.457 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T19:19:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50900a0a90 msgr2=0x7f50900a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50900a0a90 0x7f50900a2f50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5098005fd0 tx=0x7f509800f640 comp rx=0 tx=0).stop 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 msgr2=0x7f50ac1a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 0x7f50ac1a0490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f50a400ef30 tx=0x7f50a400c3a0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 shutdown_connections 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f50900a0a90 0x7f50900a2f50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac1028e0 0x7f50ac1a0490 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac100d50 0x7f50ac198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 --2- 172.21.15.86:0/522626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac101f40 0x7f50ac199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 >> 172.21.15.86:0/522626131 conn(0x7f50ac0fc2c0 msgr2=0x7f50ac072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 shutdown_connections 2024-03-22T19:19:31.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:31.452+0000 7f50b2600700 1 -- 172.21.15.86:0/522626131 wait complete. 2024-03-22T19:19:31.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:31 smithi131 ceph-mon[78559]: pgmap v542: 97 pgs: 97 active+clean; 530 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 238 op/s 2024-03-22T19:19:31.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:31 smithi086 ceph-mon[38032]: pgmap v542: 97 pgs: 97 active+clean; 530 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 238 op/s 2024-03-22T19:19:31.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:31 smithi167 ceph-mon[47728]: pgmap v542: 97 pgs: 97 active+clean; 530 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 238 op/s 2024-03-22T19:19:32.471 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-22T19:19:32.472 INFO:tasks.fwd_scrub:done joining 2024-03-22T19:19:32.472 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-22T19:19:32.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 -- 172.21.15.86:0/1326277426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48101e40 msgr2=0x7fde481022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 --2- 172.21.15.86:0/1326277426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48101e40 0x7fde481022a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde340099c0 tx=0x7fde34009cd0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 -- 172.21.15.86:0/1326277426 shutdown_connections 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 --2- 172.21.15.86:0/1326277426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde481027e0 0x7fde48109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 --2- 172.21.15.86:0/1326277426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde48100c50 0x7fde48101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 --2- 172.21.15.86:0/1326277426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48101e40 0x7fde481022a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 -- 172.21.15.86:0/1326277426 >> 172.21.15.86:0/1326277426 conn(0x7fde480fc1c0 msgr2=0x7fde480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 -- 172.21.15.86:0/1326277426 shutdown_connections 2024-03-22T19:19:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.685+0000 7fde4fa00700 1 -- 172.21.15.86:0/1326277426 wait complete. 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 Processor -- start 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 -- start start 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 0x7fde48074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde48101e40 0x7fde480731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde481027e0 0x7fde480736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48073e80 con 0x7fde48100c50 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48073ff0 con 0x7fde48101e40 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.686+0000 7fde4fa00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde481a0c30 con 0x7fde481027e0 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 0x7fde48074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 0x7fde48074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37394/0 (socket says 172.21.15.86:37394) 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde48101e40 0x7fde480731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 -- 172.21.15.86:0/2607592396 learned_addr learned my addr 172.21.15.86:0/2607592396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4da00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde481027e0 0x7fde480736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 -- 172.21.15.86:0/2607592396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde481027e0 msgr2=0x7fde480736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde481027e0 0x7fde480736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 -- 172.21.15.86:0/2607592396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde48101e40 msgr2=0x7fde480731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde48101e40 0x7fde480731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 -- 172.21.15.86:0/2607592396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340095f0 con 0x7fde48100c50 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4da00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde481027e0 0x7fde480736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.687+0000 7fde4d000700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 0x7fde48074b10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00e970 tx=0x7fde3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.688+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c009790 con 0x7fde48100c50 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.688+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde3c00eec0 con 0x7fde48100c50 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.688+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde481a0ec0 con 0x7fde48100c50 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.688+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c003c90 con 0x7fde48100c50 2024-03-22T19:19:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.688+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde481a12e0 con 0x7fde48100c50 2024-03-22T19:19:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.689+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde481080a0 con 0x7fde48100c50 2024-03-22T19:19:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.691+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fde3c00cc70 con 0x7fde48100c50 2024-03-22T19:19:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.692+0000 7fde45600700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde300a0a90 0x7fde300a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.692+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fde3c014070 con 0x7fde48100c50 2024-03-22T19:19:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.692+0000 7fde47e00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde300a0a90 0x7fde300a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.693+0000 7fde47e00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde300a0a90 0x7fde300a2f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fde34003730 tx=0x7fde3400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.696+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fde3c095c50 con 0x7fde48100c50 2024-03-22T19:19:32.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.940+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde480679e0 con 0x7fde48100c50 2024-03-22T19:19:32.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.941+0000 7fde45600700 1 -- 172.21.15.86:0/2607592396 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7fde3c099270 con 0x7fde48100c50 2024-03-22T19:19:32.943 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:19:32.943 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:19:32.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde300a0a90 msgr2=0x7fde300a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:32.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde300a0a90 0x7fde300a2f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fde34003730 tx=0x7fde3400b540 comp rx=0 tx=0).stop 2024-03-22T19:19:32.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 msgr2=0x7fde48074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 0x7fde48074b10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00e970 tx=0x7fde3c00ec80 comp rx=0 tx=0).stop 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 shutdown_connections 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fde300a0a90 0x7fde300a2f50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde481027e0 0x7fde480736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde48100c50 0x7fde48074b10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 --2- 172.21.15.86:0/2607592396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde48101e40 0x7fde480731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.944+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 >> 172.21.15.86:0/2607592396 conn(0x7fde480fc1c0 msgr2=0x7fde481051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.945+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 shutdown_connections 2024-03-22T19:19:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:32.945+0000 7fde4fa00700 1 -- 172.21.15.86:0/2607592396 wait complete. 2024-03-22T19:19:32.964 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:19:32.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:19:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 -- 172.21.15.86:0/3583323645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf8101f30 msgr2=0x7f0cf8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 --2- 172.21.15.86:0/3583323645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf8101f30 0x7f0cf8102390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40099c0 tx=0x7f0ce4009cd0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 -- 172.21.15.86:0/3583323645 shutdown_connections 2024-03-22T19:19:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 --2- 172.21.15.86:0/3583323645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf81028d0 0x7f0cf8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 --2- 172.21.15.86:0/3583323645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 --2- 172.21.15.86:0/3583323645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf8101f30 0x7f0cf8102390 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 -- 172.21.15.86:0/3583323645 >> 172.21.15.86:0/3583323645 conn(0x7f0cf80fc2b0 msgr2=0x7f0cf80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 -- 172.21.15.86:0/3583323645 shutdown_connections 2024-03-22T19:19:33.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.165+0000 7f0cfe000700 1 -- 172.21.15.86:0/3583323645 wait complete. 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.166+0000 7f0cfe000700 1 Processor -- start 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.166+0000 7f0cfe000700 1 -- start start 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cfe000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cfe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8101f30 0x7f0cf80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cfe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf81028d0 0x7f0cf80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cfe000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8073e80 con 0x7f0cf81028d0 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cfe000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8073ff0 con 0x7f0cf8101f30 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cfe000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a0d20 con 0x7f0cf8100d40 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf81028d0 0x7f0cf80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:42812/0 (socket says 172.21.15.86:42812) 2024-03-22T19:19:33.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7400700 1 -- 172.21.15.86:0/2238730023 learned_addr learned my addr 172.21.15.86:0/2238730023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf6a00700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8101f30 0x7f0cf80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7e00700 1 -- 172.21.15.86:0/2238730023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 msgr2=0x7f0cf8074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7e00700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.167+0000 7f0cf7e00700 1 -- 172.21.15.86:0/2238730023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8101f30 msgr2=0x7f0cf80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cf7e00700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8101f30 0x7f0cf80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cf7e00700 1 -- 172.21.15.86:0/2238730023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40095f0 con 0x7f0cf81028d0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cf7400700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cf7e00700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf81028d0 0x7f0cf80736e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000efc0 tx=0x7f0cf000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0018070 con 0x7f0cf81028d0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cf000f040 con 0x7f0cf81028d0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf81a0f50 con 0x7f0cf81028d0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000cce0 con 0x7f0cf81028d0 2024-03-22T19:19:33.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.168+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf81a1380 con 0x7f0cf81028d0 2024-03-22T19:19:33.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.169+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf8108190 con 0x7f0cf81028d0 2024-03-22T19:19:33.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.171+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0cf0003a00 con 0x7f0cf81028d0 2024-03-22T19:19:33.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.172+0000 7f0cefe00700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0cdc0a0a90 0x7f0cdc0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.172+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f0cf00cbbb0 con 0x7f0cf81028d0 2024-03-22T19:19:33.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.173+0000 7f0cf7400700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0cdc0a0a90 0x7f0cdc0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.173+0000 7f0cf7400700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0cdc0a0a90 0x7f0cdc0a2f50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005950 tx=0x7f0ce8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:33.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.176+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0cf0095600 con 0x7f0cf81028d0 2024-03-22T19:19:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.387+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cf80619c0 con 0x7f0cf81028d0 2024-03-22T19:19:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.387+0000 7f0cefe00700 1 -- 172.21.15.86:0/2238730023 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0cf001b320 con 0x7f0cf81028d0 2024-03-22T19:19:33.388 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.389+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0cdc0a0a90 msgr2=0x7f0cdc0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.389+0000 7f0cfe000700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0cdc0a0a90 0x7f0cdc0a2f50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005950 tx=0x7f0ce8009260 comp rx=0 tx=0).stop 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.389+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf81028d0 msgr2=0x7f0cf80736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.389+0000 7f0cfe000700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf81028d0 0x7f0cf80736e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000efc0 tx=0x7f0cf000c3a0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 shutdown_connections 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f0cdc0a0a90 0x7f0cdc0a2f50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf81028d0 0x7f0cf80736e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cf8100d40 0x7f0cf8074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 --2- 172.21.15.86:0/2238730023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8101f30 0x7f0cf80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 >> 172.21.15.86:0/2238730023 conn(0x7f0cf80fc2b0 msgr2=0x7f0cf81052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 shutdown_connections 2024-03-22T19:19:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.390+0000 7f0cfe000700 1 -- 172.21.15.86:0/2238730023 wait complete. 2024-03-22T19:19:33.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 -- 172.21.15.86:0/1602606113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 msgr2=0x7f25a4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 --2- 172.21.15.86:0/1602606113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4104570 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2590009990 tx=0x7f2590009ca0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 -- 172.21.15.86:0/1602606113 shutdown_connections 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 --2- 172.21.15.86:0/1602606113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a4104ab0 0x7f25a410b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 --2- 172.21.15.86:0/1602606113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25a4102f20 0x7f25a4103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 --2- 172.21.15.86:0/1602606113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4104570 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 -- 172.21.15.86:0/1602606113 >> 172.21.15.86:0/1602606113 conn(0x7f25a40fe500 msgr2=0x7f25a4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 -- 172.21.15.86:0/1602606113 shutdown_connections 2024-03-22T19:19:33.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.610+0000 7f25a8e00700 1 -- 172.21.15.86:0/1602606113 wait complete. 2024-03-22T19:19:33.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 Processor -- start 2024-03-22T19:19:33.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 -- start start 2024-03-22T19:19:33.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25a4102f20 0x7f25a4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a4104ab0 0x7f25a41a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41a0a50 con 0x7f25a41040f0 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41a0b90 con 0x7f25a4102f20 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a8e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41a0d20 con 0x7f25a4104ab0 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.611+0000 7f25a1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37444/0 (socket says 172.21.15.86:37444) 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a2800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a4104ab0 0x7f25a41a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 -- 172.21.15.86:0/3981185976 learned_addr learned my addr 172.21.15.86:0/3981185976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25a4102f20 0x7f25a4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 -- 172.21.15.86:0/3981185976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a4104ab0 msgr2=0x7f25a41a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a4104ab0 0x7f25a41a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 -- 172.21.15.86:0/3981185976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25a4102f20 msgr2=0x7f25a4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25a4102f20 0x7f25a4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 -- 172.21.15.86:0/3981185976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900095f0 con 0x7f25a41040f0 2024-03-22T19:19:33.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a1400700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4199460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2590000c00 tx=0x7f2590005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259001d070 con 0x7f25a41040f0 2024-03-22T19:19:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2590021410 con 0x7f25a41040f0 2024-03-22T19:19:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.612+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a41a0fa0 con 0x7f25a41040f0 2024-03-22T19:19:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.613+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590017650 con 0x7f25a41040f0 2024-03-22T19:19:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.613+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a41a1360 con 0x7f25a41040f0 2024-03-22T19:19:33.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.614+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a410a4b0 con 0x7f25a41040f0 2024-03-22T19:19:33.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.617+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f2590021580 con 0x7f25a41040f0 2024-03-22T19:19:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.618+0000 7f259aa00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25880a0a90 0x7f25880a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.618+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f25900cd9b0 con 0x7f25a41040f0 2024-03-22T19:19:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.618+0000 7f25a1e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25880a0a90 0x7f25880a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.619+0000 7f25a1e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25880a0a90 0x7f25880a2f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f25a4102c40 tx=0x7f2594007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.620+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f25900974a0 con 0x7f25a41040f0 2024-03-22T19:19:33.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:33 smithi131 ceph-mon[78559]: pgmap v543: 97 pgs: 97 active+clean; 101 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 200 op/s 2024-03-22T19:19:33.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2607592396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:19:33.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:33 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2238730023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:19:33.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:33 smithi086 ceph-mon[38032]: pgmap v543: 97 pgs: 97 active+clean; 101 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 200 op/s 2024-03-22T19:19:33.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2607592396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:19:33.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:33 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2238730023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:19:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:33 smithi167 ceph-mon[47728]: pgmap v543: 97 pgs: 97 active+clean; 101 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 200 op/s 2024-03-22T19:19:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2607592396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:19:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:33 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2238730023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:19:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.835+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25a402d120 con 0x7f25a41040f0 2024-03-22T19:19:33.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.835+0000 7f259aa00700 1 -- 172.21.15.86:0/3981185976 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f259009aac0 con 0x7f25a41040f0 2024-03-22T19:19:33.836 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25880a0a90 msgr2=0x7f25880a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25880a0a90 0x7f25880a2f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f25a4102c40 tx=0x7f2594007400 comp rx=0 tx=0).stop 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 msgr2=0x7f25a4199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4199460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2590000c00 tx=0x7f2590005740 comp rx=0 tx=0).stop 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 shutdown_connections 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f25880a0a90 0x7f25880a2f50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a4104ab0 0x7f25a41a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25a4102f20 0x7f25a4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 --2- 172.21.15.86:0/3981185976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25a41040f0 0x7f25a4199460 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 >> 172.21.15.86:0/3981185976 conn(0x7f25a40fe500 msgr2=0x7f25a41075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 shutdown_connections 2024-03-22T19:19:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:33.838+0000 7f25a8e00700 1 -- 172.21.15.86:0/3981185976 wait complete. 2024-03-22T19:19:33.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:19:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 -- 172.21.15.86:0/449280529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964101d70 msgr2=0x7f09641021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 --2- 172.21.15.86:0/449280529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964101d70 0x7f09641021d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0950009920 tx=0x7f0950009c30 comp rx=0 tx=0).stop 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 -- 172.21.15.86:0/449280529 shutdown_connections 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 --2- 172.21.15.86:0/449280529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964102710 0x7f0964108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 --2- 172.21.15.86:0/449280529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964100b80 0x7f0964100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 --2- 172.21.15.86:0/449280529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964101d70 0x7f09641021d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 -- 172.21.15.86:0/449280529 >> 172.21.15.86:0/449280529 conn(0x7f09640fc0f0 msgr2=0x7f09640fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 -- 172.21.15.86:0/449280529 shutdown_connections 2024-03-22T19:19:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.058+0000 7f096a200700 1 -- 172.21.15.86:0/449280529 wait complete. 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.059+0000 7f096a200700 1 Processor -- start 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.059+0000 7f096a200700 1 -- start start 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f096a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964100b80 0x7f0964074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f096a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 0x7f09640731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f096a200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964102710 0x7f09640736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f096a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964073e80 con 0x7f0964101d70 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f096a200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964073ff0 con 0x7f0964100b80 2024-03-22T19:19:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f096a200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09641a0c20 con 0x7f0964102710 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 0x7f09640731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 0x7f09640731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37468/0 (socket says 172.21.15.86:37468) 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 -- 172.21.15.86:0/724492355 learned_addr learned my addr 172.21.15.86:0/724492355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0968e00700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964100b80 0x7f0964074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0969800700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964102710 0x7f09640736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 -- 172.21.15.86:0/724492355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964102710 msgr2=0x7f09640736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964102710 0x7f09640736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 -- 172.21.15.86:0/724492355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964100b80 msgr2=0x7f0964074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964100b80 0x7f0964074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.060+0000 7f0963e00700 1 -- 172.21.15.86:0/724492355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580095f0 con 0x7f0964101d70 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.061+0000 7f0963e00700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 0x7f09640731a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0950009840 tx=0x7f095000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.061+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095001c070 con 0x7f0964101d70 2024-03-22T19:19:34.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.061+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095000f7d0 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.061+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0950009520 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.061+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950003bb0 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.061+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09641a10e0 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.062+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09641080a0 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.064+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f0950003d50 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.066+0000 7f0961600700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f094c0a0b10 0x7f094c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.066+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f09500ccd60 con 0x7f0964101d70 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.066+0000 7f0968e00700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f094c0a0b10 0x7f094c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.067+0000 7f0968e00700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f094c0a0b10 0x7f094c0a2fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0964101bd0 tx=0x7f0958009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.070+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f09500967b0 con 0x7f0964101d70 2024-03-22T19:19:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.278+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09640619c0 con 0x7f0964101d70 2024-03-22T19:19:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.278+0000 7f0961600700 1 -- 172.21.15.86:0/724492355 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0950099dd0 con 0x7f0964101d70 2024-03-22T19:19:34.279 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:19:34.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f094c0a0b10 msgr2=0x7f094c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f094c0a0b10 0x7f094c0a2fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0964101bd0 tx=0x7f0958009310 comp rx=0 tx=0).stop 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 msgr2=0x7f09640731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 0x7f09640731a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0950009840 tx=0x7f095000b790 comp rx=0 tx=0).stop 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 shutdown_connections 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f094c0a0b10 0x7f094c0a2fd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964102710 0x7f09640736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0964100b80 0x7f0964074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 --2- 172.21.15.86:0/724492355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0964101d70 0x7f09640731a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 >> 172.21.15.86:0/724492355 conn(0x7f09640fc0f0 msgr2=0x7f09641051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 shutdown_connections 2024-03-22T19:19:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.281+0000 7f096a200700 1 -- 172.21.15.86:0/724492355 wait complete. 2024-03-22T19:19:34.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-22T19:19:34.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 -- 172.21.15.86:0/1947846437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 msgr2=0x7f1864101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 --2- 172.21.15.86:0/1947846437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864101070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f18540099c0 tx=0x7f1854009cd0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 -- 172.21.15.86:0/1947846437 shutdown_connections 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 --2- 172.21.15.86:0/1947846437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18641027e0 0x7f1864109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 --2- 172.21.15.86:0/1947846437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864101070 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 --2- 172.21.15.86:0/1947846437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1864101e40 0x7f18641022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.492+0000 7f1869a00700 1 -- 172.21.15.86:0/1947846437 >> 172.21.15.86:0/1947846437 conn(0x7f18640fc1c0 msgr2=0x7f18640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.493+0000 7f1869a00700 1 -- 172.21.15.86:0/1947846437 shutdown_connections 2024-03-22T19:19:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.493+0000 7f1869a00700 1 -- 172.21.15.86:0/1947846437 wait complete. 2024-03-22T19:19:34.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.493+0000 7f1869a00700 1 Processor -- start 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 -- start start 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1864101e40 0x7f18640731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641027e0 0x7f18640736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864073e80 con 0x7f1864100c50 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864073ff0 con 0x7f1864101e40 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1869a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a0c60 con 0x7f18641027e0 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1862a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1862a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37488/0 (socket says 172.21.15.86:37488) 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1862a00700 1 -- 172.21.15.86:0/605059227 learned_addr learned my addr 172.21.15.86:0/605059227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1862000700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1864101e40 0x7f18640731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.494+0000 7f1863400700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641027e0 0x7f18640736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1862a00700 1 -- 172.21.15.86:0/605059227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641027e0 msgr2=0x7f18640736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1862a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641027e0 0x7f18640736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1862a00700 1 -- 172.21.15.86:0/605059227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1864101e40 msgr2=0x7f18640731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1862a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1864101e40 0x7f18640731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1862a00700 1 -- 172.21.15.86:0/605059227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540095f0 con 0x7f1864100c50 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1863400700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641027e0 0x7f18640736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1862a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864074b10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1854003730 tx=0x7f185400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185401d070 con 0x7f1864100c50 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1854005050 con 0x7f1864100c50 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18641a0e90 con 0x7f1864100c50 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854004af0 con 0x7f1864100c50 2024-03-22T19:19:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.495+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641a1200 con 0x7f1864100c50 2024-03-22T19:19:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.496+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1864108120 con 0x7f1864100c50 2024-03-22T19:19:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.498+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f18540051c0 con 0x7f1864100c50 2024-03-22T19:19:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.500+0000 7f185b400700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18480a0a90 0x7f18480a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:19:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.500+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f18540cd200 con 0x7f1864100c50 2024-03-22T19:19:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.500+0000 7f1862000700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18480a0a90 0x7f18480a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:19:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.501+0000 7f1862000700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18480a0a90 0x7f18480a2f50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1850009c60 tx=0x7f1850009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:19:34.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.502+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1854096c50 con 0x7f1864100c50 2024-03-22T19:19:34.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:34 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3981185976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:19:34.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:34 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/724492355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:19:34.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:34 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3981185976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:19:34.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:34 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/724492355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:19:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.765+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f18640679e0 con 0x7f1864100c50 2024-03-22T19:19:34.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.776+0000 7f185b400700 1 -- 172.21.15.86:0/605059227 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+731465 (secure 0 0 0) 0x7f185409a270 con 0x7f1864100c50 2024-03-22T19:19:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:19:34.787 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:19:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 18, 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 14152, 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1126645338 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-03-22T19:19:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1126645338 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/1126645338", 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-03-22T19:03:50.084202+0000", 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540138322906710015, 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24109, 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540138322906710015, 2024-03-22T19:19:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-03-22T19:19:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-03-22T19:19:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-03-22T19:19:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-03-22T19:19:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-03-22T19:19:34.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.804 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-03-22T19:19:34.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.819 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap" 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-03-22T19:19:34.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.826 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.827 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-22T19:19:34.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:34.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-03-22T19:19:34.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-03-22T19:19:34.837 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.840 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-22T19:19:34.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-03-22T19:19:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.850 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:34.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-03-22T19:19:34.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-03-22T19:19:34.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-03-22T19:19:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-03-22T19:19:34.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-03-22T19:19:34.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.864 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-22T19:19:34.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-22T19:19:34.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-03-22T19:19:34.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-03-22T19:19:34.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-03-22T19:19:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-22T19:19:34.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-22T19:19:34.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.874 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-22T19:19:34.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-22T19:19:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-22T19:19:34.878 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-22T19:19:34.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.881 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.882 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.888 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.890 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-03-22T19:19:34.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-03-22T19:19:34.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.902 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.908 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.910 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.920 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-22T19:19:34.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.926 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-22T19:19:34.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-22T19:19:34.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-22T19:19:34.929 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-03-22T19:19:34.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-22T19:19:34.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-22T19:19:34.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-22T19:19:34.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-22T19:19:34.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.935 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-22T19:19:34.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-03-22T19:19:34.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.944 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.944 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.944 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.944 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.945 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.945 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-22T19:19:34.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.964 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-03-22T19:19:34.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-03-22T19:19:34.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:34.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.972 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.977 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-03-22T19:19:34.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-03-22T19:19:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.981 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-03-22T19:19:34.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.985 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.991 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-03-22T19:19:34.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:34.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:34.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:34.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:34.999 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.001 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.005 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.008 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.010 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.013 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-03-22T19:19:35.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.020 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-03-22T19:19:35.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.027 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-22T19:19:35.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.033 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.037 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.039 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.041 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.042 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-03-22T19:19:35.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-03-22T19:19:35.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-03-22T19:19:35.060 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.061 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.069 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-03-22T19:19:35.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.076 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-03-22T19:19:35.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-03-22T19:19:35.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-03-22T19:19:35.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.086 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.100 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.114 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.126 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.127 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-03-22T19:19:35.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.130 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.131 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.135 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.137 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.138 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-22T19:19:35.148 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-03-22T19:19:35.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.151 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.154 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.155 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.157 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.165 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.167 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.168 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.170 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-22T19:19:35.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-22T19:19:35.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-22T19:19:35.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.189 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.190 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.191 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.196 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.197 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.200 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.202 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.205 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.213 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24115, 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540138322906710015, 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-03-22T19:19:35.221 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-03-22T19:19:35.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.223 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-03-22T19:19:35.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.232 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.237 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-03-22T19:19:35.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-03-22T19:19:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-03-22T19:19:35.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-03-22T19:19:35.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.243 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.244 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.245 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.250 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap" 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.260 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.264 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-22T19:19:35.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-22T19:19:35.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-22T19:19:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-22T19:19:35.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-22T19:19:35.272 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-22T19:19:35.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-22T19:19:35.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-22T19:19:35.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-22T19:19:35.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-03-22T19:19:35.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.290 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-03-22T19:19:35.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.293 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.294 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.295 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-03-22T19:19:35.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.298 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.301 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.303 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-03-22T19:19:35.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-03-22T19:19:35.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-03-22T19:19:35.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-03-22T19:19:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-03-22T19:19:35.311 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-03-22T19:19:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-03-22T19:19:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.314 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.315 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.320 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-03-22T19:19:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.324 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.330 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.340 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-22T19:19:35.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-22T19:19:35.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.371 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-03-22T19:19:35.374 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.376 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.377 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.378 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.380 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.384 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.387 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.390 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-03-22T19:19:35.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.394 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.395 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.396 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.399 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.400 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-22T19:19:35.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-03-22T19:19:35.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-22T19:19:35.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-03-22T19:19:35.405 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.409 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.411 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.412 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.414 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-03-22T19:19:35.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.419 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.422 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-03-22T19:19:35.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-03-22T19:19:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-03-22T19:19:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-03-22T19:19:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-03-22T19:19:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-03-22T19:19:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.478 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.478 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.478 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.478 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.479 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.480 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.480 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.480 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.480 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.481 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.481 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.483 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.487 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.488 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.488 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.489 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.493 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.494 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.495 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-22T19:19:35.497 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.498 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.498 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.498 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-03-22T19:19:35.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-03-22T19:19:35.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.499 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.500 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.503 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.504 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-03-22T19:19:35.507 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.513 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.515 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-03-22T19:19:35.518 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.518 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.518 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.518 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.520 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.521 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.522 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.523 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.526 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.528 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.538 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-03-22T19:19:35.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-03-22T19:19:35.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.547 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.550 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-22T19:19:35.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-03-22T19:19:35.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.559 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-22T19:19:35.563 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.564 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-03-22T19:19:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-03-22T19:19:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-03-22T19:19:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-03-22T19:19:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-03-22T19:19:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-03-22T19:19:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.631 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.632 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.634 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.640 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-03-22T19:19:35.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-03-22T19:19:35.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-03-22T19:19:35.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.659 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.663 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.664 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.668 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.669 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-03-22T19:19:35.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-03-22T19:19:35.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.678 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.681 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "restful" 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.685 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.690 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.694 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-03-22T19:19:35.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-03-22T19:19:35.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.702 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.703 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.704 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-03-22T19:19:35.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-03-22T19:19:35.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-03-22T19:19:35.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-22T19:19:35.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-03-22T19:19:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap" 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-22T19:19:35.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-22T19:19:35.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-03-22T19:19:35.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-03-22T19:19:35.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.722 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.723 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.724 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.725 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.732 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.737 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-03-22T19:19:35.741 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-22T19:19:35.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.746 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.750 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.754 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.755 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.762 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-22T19:19:35.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.789 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.790 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:35.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-22T19:19:35.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-03-22T19:19:35.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-03-22T19:19:35.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.840 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.841 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.842 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-22T19:19:35.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-03-22T19:19:35.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.853 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.855 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-03-22T19:19:35.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-03-22T19:19:35.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.864 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.866 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.869 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-03-22T19:19:35.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.879 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.880 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.881 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.882 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-03-22T19:19:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.892 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.894 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.895 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.898 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-22T19:19:35.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.902 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.905 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.907 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-03-22T19:19:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-22T19:19:35.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.913 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-03-22T19:19:35.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.921 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.923 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.926 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.927 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.929 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-03-22T19:19:35.930 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.931 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.937 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.938 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.939 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.946 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.952 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.955 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.958 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-03-22T19:19:35.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.967 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:35.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.976 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.977 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-22T19:19:35.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.988 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.991 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-03-22T19:19:35.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-03-22T19:19:35.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:35.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.997 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:35.998 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:35.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.006 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.008 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-22T19:19:36.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.020 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-03-22T19:19:36.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-03-22T19:19:36.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-03-22T19:19:36.024 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-03-22T19:19:36.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-22T19:19:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.034 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-03-22T19:19:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.037 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.039 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.043 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-03-22T19:19:36.047 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.048 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.049 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:36.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.058 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-22T19:19:36.062 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-22T19:19:36.064 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-22T19:19:36.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.074 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.078 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.079 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.084 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.085 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.088 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.090 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.091 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-22T19:19:36.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-03-22T19:19:36.095 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.099 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-22T19:19:36.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-03-22T19:19:36.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-22T19:19:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-22T19:19:36.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-03-22T19:19:36.106 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-22T19:19:36.107 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-22T19:19:36.108 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-22T19:19:36.109 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-22T19:19:36.110 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 3, 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-03-22T19:19:36.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1673116221 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 375941103 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2025404564 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:36.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4038820216 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T19:19:36.115 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:19:36.117 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:35 smithi131 ceph-mon[78559]: pgmap v544: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 173 op/s 2024-03-22T19:19:36.117 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:35 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/605059227' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-22T19:19:36.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:34 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3981185976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:19:36.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:34 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/724492355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:19:36.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:35 smithi167 ceph-mon[47728]: pgmap v544: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 173 op/s 2024-03-22T19:19:36.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:35 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/605059227' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-22T19:19:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18480a0a90 msgr2=0x7f18480a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18480a0a90 0x7f18480a2f50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1850009c60 tx=0x7f1850009260 comp rx=0 tx=0).stop 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 msgr2=0x7f1864074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864074b10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1854003730 tx=0x7f185400f690 comp rx=0 tx=0).stop 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 shutdown_connections 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f18480a0a90 0x7f18480a2f50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641027e0 0x7f18640736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1864100c50 0x7f1864074b10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 --2- 172.21.15.86:0/605059227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1864101e40 0x7f18640731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.783+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 >> 172.21.15.86:0/605059227 conn(0x7f18640fc1c0 msgr2=0x7f1864105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.784+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 shutdown_connections 2024-03-22T19:19:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:19:34.784+0000 7f1869a00700 1 -- 172.21.15.86:0/605059227 wait complete. 2024-03-22T19:19:36.125 DEBUG:tasks.check_counter:Getting stats from a 2024-03-22T19:19:36.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-22T19:19:36.125 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:35 smithi086 ceph-mon[38032]: pgmap v544: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 173 op/s 2024-03-22T19:19:36.126 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:35 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/605059227' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-22T19:19:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:19:37.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:37 smithi086 ceph-mon[38032]: pgmap v545: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 134 op/s 2024-03-22T19:19:37.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:37 smithi131 ceph-mon[78559]: pgmap v545: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 134 op/s 2024-03-22T19:19:37.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:37 smithi167 ceph-mon[47728]: pgmap v545: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 134 op/s 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1201, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1085, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1259328, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 3654078, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 15, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.211009341, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.068344699, 2024-03-22T19:19:38.407 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.090796967, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.048676934, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1084, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.324354853, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000299220 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 498, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000116238, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000233 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 759712, 2024-03-22T19:19:38.408 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 109536 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1473, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1527, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 807586, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 931483, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 10, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.219625135, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.082596619, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.107240563, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.032353056, 2024-03-22T19:19:38.409 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1527, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.372541877, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000243969 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1740, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000548834, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000315 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 307344 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.410 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3051, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 3715, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1008065, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1582865, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 29, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 10, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.054078138, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.771254102, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.267786718, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.040169163, 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:38.411 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3710, 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.713445927, 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000461845 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2203, 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001785798, 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000810 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.414 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1252, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.376123902, 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001099140 2024-03-22T19:19:38.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 860, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.056855521, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000066111 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 361, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 355, 2024-03-22T19:19:38.416 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 355, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.621481337, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001750651 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 6, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 11, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 17, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-03-22T19:19:38.417 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 17, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 13, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 18, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 4, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 473, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 446, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 5, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 9, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-22T19:19:38.418 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 5, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 5, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 292, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 778, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 324, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 13424350788, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 778, 2024-03-22T19:19:38.419 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 292, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 292, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 486, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 171, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 455, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:19:38.420 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 1797, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 1446, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 14, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 365 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 3, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 2, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 12, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-22T19:19:38.421 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 199, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 199, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 5, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:19:38.422 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 2174, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 2174, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 3, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 3, 2024-03-22T19:19:38.423 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 7386448, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 254, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 14.537364921, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.057233720 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 22, 2024-03-22T19:19:38.424 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 230, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 208, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 21, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 104, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 83, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 17, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 526, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 509, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 244, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 244, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 55960, 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 198964 2024-03-22T19:19:38.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 588, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 6, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 361, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 10, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 20, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2, 2024-03-22T19:19:38.426 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000928985, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000464492 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-22T19:19:38.427 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 82, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.031362104, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000382464 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.428 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.430 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-22T19:19:38.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 72, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.027284873, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000378956 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 73, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.424164042, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005810466 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.433 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.434 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 126, 2024-03-22T19:19:38.436 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.137741333, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001093185 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-03-22T19:19:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 324, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 324, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 421, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 300457, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 122, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:38.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 7344, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 155, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 87245, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 350, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-22T19:19:38.442 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 1800, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 3248370, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 27, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 1800, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1800, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 314.444143179, 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.174691190 2024-03-22T19:19:38.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1800, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1861 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 1800, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 782, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 1018, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-03-22T19:19:38.444 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 4, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 258, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 32, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 669, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 778, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:19:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-22T19:19:38.446 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 82, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-22T19:19:38.447 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 2, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 68, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 22 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-22T19:19:38.448 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 372 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 51, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 6, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 199, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:38.449 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 5721, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2582761, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 5721, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5721, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2582761, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1800, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3268469, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1800, 2024-03-22T19:19:38.451 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3268469, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-22T19:19:38.452 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1800, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1800, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1800, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1800, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:38.453 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 199, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 20099, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 199, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 20099, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55cf541720a0": { 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2174, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 3192144, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2174, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 254, 2024-03-22T19:19:38.455 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3192144, 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:38.456 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:19:38.847 INFO:tasks.check_counter:Daemon mds.a mds.exported=5 2024-03-22T19:19:38.847 INFO:tasks.check_counter:Daemon mds.a mds.imported=5 2024-03-22T19:19:38.847 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-03-22T19:19:38.847 DEBUG:tasks.check_counter:Getting stats from d 2024-03-22T19:19:38.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-22T19:19:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:19:39.706 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:39 smithi086 ceph-mon[38032]: pgmap v546: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 86 op/s 2024-03-22T19:19:39.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:39 smithi131 ceph-mon[78559]: pgmap v546: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 86 op/s 2024-03-22T19:19:39.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:39 smithi167 ceph-mon[47728]: pgmap v546: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 86 op/s 2024-03-22T19:19:41.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:41 smithi131 ceph-mon[78559]: pgmap v547: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 60 op/s 2024-03-22T19:19:41.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:41 smithi086 ceph-mon[38032]: pgmap v547: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 60 op/s 2024-03-22T19:19:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:41 smithi167 ceph-mon[47728]: pgmap v547: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 60 op/s 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-03-22T19:19:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.003806776, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000839078, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 179, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 176, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 459200, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 99888, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-03-22T19:19:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.047105324, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.023623653, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.016977809, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.006424329, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 175, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.028944225, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000165395 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 459200, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 99888 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 5, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 135, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3104, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 99808, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 3, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.016606082, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.013012725, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002261958, 2024-03-22T19:19:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 134, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.022572470, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000168451 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 99808 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 149998, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 30, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:42.971 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 180, 2024-03-22T19:19:42.972 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 433933, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 180, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 180, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 433933, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:42.973 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:19:43.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:43 smithi131 ceph-mon[78559]: pgmap v548: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 22 op/s 2024-03-22T19:19:43.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:43 smithi086 ceph-mon[38032]: pgmap v548: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 22 op/s 2024-03-22T19:19:43.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:43 smithi167 ceph-mon[47728]: pgmap v548: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 22 op/s 2024-03-22T19:19:45.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:45 smithi131 ceph-mon[78559]: pgmap v549: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T19:19:45.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:45 smithi086 ceph-mon[38032]: pgmap v549: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T19:19:45.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:45 smithi167 ceph-mon[47728]: pgmap v549: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T19:19:47.681 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:47 smithi086 ceph-mon[38032]: pgmap v550: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-22T19:19:47.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:47 smithi131 ceph-mon[78559]: pgmap v550: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-22T19:19:47.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:47 smithi167 ceph-mon[47728]: pgmap v550: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-22T19:19:49.693 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:49 smithi086 ceph-mon[38032]: pgmap v551: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-22T19:19:49.701 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-22T19:19:49.701 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-22T19:19:49.701 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-03-22T19:19:49.701 DEBUG:tasks.check_counter:Getting stats from g 2024-03-22T19:19:49.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-22T19:19:49.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:49 smithi131 ceph-mon[78559]: pgmap v551: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-22T19:19:49.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:49 smithi167 ceph-mon[47728]: pgmap v551: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-22T19:19:50.074 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:19:51.303 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 569, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 571, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 191129, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 176960, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.106712327, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.046224900, 2024-03-22T19:19:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.019505170, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.030746092, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 569, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.091833790, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000161395 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1386, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1361, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2468784, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 467313, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-03-22T19:19:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.197427921, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.060927227, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.058253044, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.071560476, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1361, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.214830588, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000157847 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 598560, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 101520 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 4, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 136, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1508, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 306816, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.017396485, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.013454465, 2024-03-22T19:19:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002416970, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000071646, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 134, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.033285845, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000248401 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:19:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 305872 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2921, 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.141697761, 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000048510 2024-03-22T19:19:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-03-22T19:19:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-22T19:19:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 19, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 19, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 19, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:19:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-03-22T19:19:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:19:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-03-22T19:19:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-22T19:19:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 1265, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-22T19:19:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 5766080, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 5764429, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 1265 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 4, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 32, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 28, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 2, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 14, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 12, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 19, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 47, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 28, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-03-22T19:19:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 49560, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 207156 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-22T19:19:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.327 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.329 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-22T19:19:51.330 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.331 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-22T19:19:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1972, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:51.335 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 197503, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 55, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 7344, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 155, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:51.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 25168, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 104, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 1748, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-03-22T19:19:51.337 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 1748, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1748, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 73.143398083, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.041844049 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1748, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1749 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 1748, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 1748, 2024-03-22T19:19:51.338 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1054, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 692, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-22T19:19:51.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-22T19:19:51.340 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 82, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-22T19:19:51.341 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 2, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 6, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 6, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-22T19:19:51.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:51.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 1955, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2472040, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 1955, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1955, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2472040, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1748, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1571781, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1748, 2024-03-22T19:19:51.345 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1571781, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-22T19:19:51.346 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1748, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1748, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1748, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1748, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:51.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.348 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55e10b97c5a0": { 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:51.349 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:51.350 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:51.350 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:51.350 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.350 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:51.350 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:19:51.574 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:51 smithi086 ceph-mon[38032]: pgmap v552: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-22T19:19:51.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:51 smithi131 ceph-mon[78559]: pgmap v552: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-22T19:19:51.789 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-22T19:19:51.789 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-22T19:19:51.789 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-03-22T19:19:51.789 DEBUG:tasks.check_counter:Getting stats from j 2024-03-22T19:19:51.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-22T19:19:51.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:51 smithi167 ceph-mon[47728]: pgmap v552: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-22T19:19:52.151 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.a/config 2024-03-22T19:19:53.458 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 187, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 182, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 463232, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 102704, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.058979697, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.029261122, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.019858486, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.004904276, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 182, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.042916386, 2024-03-22T19:19:53.459 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000235804 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 463232, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 102704 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 136, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 160, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 99888, 2024-03-22T19:19:53.460 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 3, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.016633991, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.013424141, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.001682393, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 134, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.029151592, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000217549 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:53.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 99888 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.001052904, 2024-03-22T19:19:53.462 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000892270, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.463 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:53.464 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:53.465 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148967, 2024-03-22T19:19:53.466 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 29, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-22T19:19:53.467 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 184, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 434453, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 184, 2024-03-22T19:19:53.468 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 184, 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 434453, 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T19:19:53.469 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T19:19:53.704 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:53 smithi086 ceph-mon[38032]: pgmap v553: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-22T19:19:53.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:53 smithi131 ceph-mon[78559]: pgmap v553: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-22T19:19:53.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:53 smithi167 ceph-mon[47728]: pgmap v553: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-22T19:19:53.907 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-22T19:19:53.907 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-22T19:19:53.907 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-22T19:19:53.907 DEBUG:tasks.check_counter:Getting stats from b 2024-03-22T19:19:53.908 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-22T19:19:54.601 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:19:55.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:55 smithi131 ceph-mon[78559]: pgmap v554: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2024-03-22T19:19:55.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:55 smithi086 ceph-mon[38032]: pgmap v554: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2024-03-22T19:19:55.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:55 smithi167 ceph-mon[47728]: pgmap v554: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2024-03-22T19:19:56.104 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-03-22T19:19:56.104 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:56.104 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 8429, 2024-03-22T19:19:56.104 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 11111, 2024-03-22T19:19:56.104 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 2741008, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 14944395, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 96, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.589821555, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.215898095, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.391344159, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.078953240, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 11108, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2.745091440, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000247127 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 6969, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002656633, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000381 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 62288, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 32144 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 3011, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2982, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1736121, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 3136980, 2024-03-22T19:19:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.242814446, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.071505393, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.109833446, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.065941294, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2983, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.826713995, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000277141 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:56.107 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2827, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000564660, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000199 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 627008, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 117056 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2215, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2436, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 827454, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 1356219, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-03-22T19:19:56.108 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.185646562, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.062819806, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.078090483, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.044663770, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2434, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.892875205, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000366834 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2773, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000639205, 2024-03-22T19:19:56.109 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000230 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 315104 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2237, 2024-03-22T19:19:56.110 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.948766432, 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000871151 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1797, 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.080261961, 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000044664 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-03-22T19:19:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 2207, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 2203, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2203, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 14.740971639, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006691317 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 4, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 12, 2024-03-22T19:19:56.112 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 85, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 2, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 83, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 84, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 340, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 2, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 2, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 3, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 3135, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 3009, 2024-03-22T19:19:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 48, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 6, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 316, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 6, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 29, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 796, 2024-03-22T19:19:56.114 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 4, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 234, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 796, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 172, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 8, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 164, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 624, 2024-03-22T19:19:56.115 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 10, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 218, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 2134, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 6, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 3497, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 2799, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 112, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 1107 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 4, 2024-03-22T19:19:56.116 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 27, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 650, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 589, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 61, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:19:56.117 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 8, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 6, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 5553, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 5553, 2024-03-22T19:19:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 7, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 7, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 14552883, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 835, 2024-03-22T19:19:56.119 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 24.923524571, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.029848532 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 25, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 845, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 820, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 20, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 127, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 107, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 85, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 1238, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 1153, 2024-03-22T19:19:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 2, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 840, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 838, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 61012, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 207156 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 2669, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 12, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 2207, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 19, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 41, 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-03-22T19:19:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 676, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.687068810, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001016373 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 231, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.062688261, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000271377 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.122 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 61, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.050233151, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000823494 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 156, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.075063697, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000481177 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.123 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000169038, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000169038 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-03-22T19:19:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.125 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 105, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.102965221, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000980621 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 48, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.011403965, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000237582 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 164, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.928500748, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005661589 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-03-22T19:19:56.127 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 113, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 11.756267518, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.104037765 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 24, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.286404117, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.011933504 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.129 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.031922081, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.004560297 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-03-22T19:19:56.130 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001067017, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000266754 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.021705371, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.003100767 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-03-22T19:19:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 13, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.008384095, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000644930 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 593, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.717128549, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001209323 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 170, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 170, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 549, 2024-03-22T19:19:56.133 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:56.134 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:56.135 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 315629, 2024-03-22T19:19:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 135, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 792, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 9, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7344, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 155, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 148778, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 1353, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 3603, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 10500071, 2024-03-22T19:19:56.138 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 37, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 3603, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3603, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 661.789223454, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.183677275 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3603, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3703 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 3603, 2024-03-22T19:19:56.139 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 834, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 2769, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 11, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 30, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 857, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 47, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 1775, 2024-03-22T19:19:56.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 804, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 22, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 6, 2024-03-22T19:19:56.141 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-03-22T19:19:56.142 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 3, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 12, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 114, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 37 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 2261 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 25, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 6, 2024-03-22T19:19:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 589, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 13651, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 4158706, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 13651, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 13651, 2024-03-22T19:19:56.145 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4158706, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:56.146 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3603, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 10559560, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3603, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 10559560, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:56.147 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3603, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 3603, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3603, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3603, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.148 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 589, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 59489, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 589, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 22, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 59489, 2024-03-22T19:19:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5587b19780a0": { 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 5553, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 10358579, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 5553, 2024-03-22T19:19:56.150 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 835, 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 10358579, 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:56.151 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-03-22T19:19:56.477 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-03-22T19:19:56.478 INFO:tasks.check_counter:Daemon mds.b mds.imported=6 2024-03-22T19:19:56.478 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-03-22T19:19:56.478 DEBUG:tasks.check_counter:Getting stats from e 2024-03-22T19:19:56.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-22T19:19:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1845, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1847, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3710449, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 573802, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.143777625, 2024-03-22T19:19:57.638 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.039511672, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.039784608, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.053633177, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1845, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.222305454, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000120490 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 3, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 138, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 716, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 308652, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.007662552, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.005219665, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.001075945, 2024-03-22T19:19:57.640 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000068652, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 136, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.029726967, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000218580 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 307968 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 215, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 190, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 599916, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 104183, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 3, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.030044317, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.009955242, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.016637645, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.004893579, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 190, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.018072002, 2024-03-22T19:19:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000095115 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 599648, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 103872 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3052, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.073154124, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000023969 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-03-22T19:19:57.645 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 200, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 200, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 200, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-03-22T19:19:57.646 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-03-22T19:19:57.647 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:19:57.648 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-03-22T19:19:57.649 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:19:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 2174, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 3, 2024-03-22T19:19:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 7386448, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 7384797, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 2174 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 4, 2024-03-22T19:19:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 221, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 217, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 2, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 94, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 92, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 200, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 513, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 313, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 49620, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 207156 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-03-22T19:19:57.653 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.657 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.661 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:19:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-03-22T19:19:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2361, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:57.666 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:57.667 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 181259, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 46, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7344, 2024-03-22T19:19:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 155, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 152592, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 828, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-03-22T19:19:57.670 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 1848, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 1848, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1848, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 65.745724803, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.035576690 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1848, 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1849 2024-03-22T19:19:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 1848, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 1848, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 1115, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 731, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-03-22T19:19:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-03-22T19:19:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-03-22T19:19:57.674 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 4, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 4, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-03-22T19:19:57.675 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-03-22T19:19:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2059, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 4112132, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2059, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2059, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4112132, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 1848, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 3192149, 2024-03-22T19:19:57.678 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1848, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3192149, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 1848, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 1848, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1848, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1848, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:57.680 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x55e09f382500": { 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:57.682 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:57.683 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:57.683 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.683 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:57.683 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-03-22T19:19:57.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:57 smithi131 ceph-mon[78559]: pgmap v555: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-22T19:19:57.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:57 smithi086 ceph-mon[38032]: pgmap v555: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-22T19:19:57.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:57 smithi167 ceph-mon[47728]: pgmap v555: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-22T19:19:58.003 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-22T19:19:58.003 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-22T19:19:58.003 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-03-22T19:19:58.003 DEBUG:tasks.check_counter:Getting stats from h 2024-03-22T19:19:58.003 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-22T19:19:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:19:59.196 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1982, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2432, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1335670, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 5073266, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.190802768, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.072616667, 2024-03-22T19:19:59.197 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.103452393, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.026305715, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2435, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.467941048, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000192172 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2806, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000573630, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000204 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 610848, 2024-03-22T19:19:59.198 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 110512 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 646, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 782, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 256697, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 668495, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 12, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.062991394, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.018092869, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.021150748, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.020162176, 2024-03-22T19:19:59.199 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 781, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.627389966, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000803316 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 94, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000018785, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000199 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 312688 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 4030, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 4218, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1414377, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 1714644, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 29, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 8, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.223834172, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.073920528, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.222724859, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.015721485, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4212, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3.701288310, 2024-03-22T19:19:59.201 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000878748 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1656, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000789426, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000476 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-03-22T19:19:59.202 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 608, 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.590172272, 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000970678 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:19:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1041, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.033644480, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000032319 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 105, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 101, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 101, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 18.872926905, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.186860662 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.204 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 4, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 17, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 2, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 2, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 13, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 13, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 38, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-03-22T19:19:59.205 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 4, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 214, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 164, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 4, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 7, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 4, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 26, 2024-03-22T19:19:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 4, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 64, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 140, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 324, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 13424350788, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 140, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:19:59.207 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 140, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 5, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 414, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 3482, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 3224, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 3, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 78 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-03-22T19:19:59.208 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 3, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 2, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 10, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 38, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 38, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:19:59.209 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 4, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 3365, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-03-22T19:19:59.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 3365, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 5, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 5, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 8419928, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 182, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 19.632128534, 2024-03-22T19:19:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.107868838 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 22, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 91, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 69, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 21, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 30, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 9, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 17, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 131, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 114, 2024-03-22T19:19:59.212 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 86, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 86, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 54032, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 198964 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 223, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 2, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 105, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 9, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 7, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.213 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.214 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 61, 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 18.795441308, 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.308121988 2024-03-22T19:19:59.215 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.216 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-03-22T19:19:59.218 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001276483, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000638241 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.219 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.222 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 38, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.076209114, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002005503 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-03-22T19:19:59.224 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 82, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 82, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 444, 2024-03-22T19:19:59.225 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:19:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:19:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 292101, 2024-03-22T19:19:59.228 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 115, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7344, 2024-03-22T19:19:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 155, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 87469, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 358, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-03-22T19:19:59.230 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 1448, 2024-03-22T19:19:59.231 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 4256571, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 27, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 1448, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1448, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 340.335580314, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.235038384 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1448, 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1542 2024-03-22T19:19:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 1448, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 141, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 1307, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 183, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 31, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-03-22T19:19:59.233 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 1015, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 140, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-03-22T19:19:59.234 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-03-22T19:19:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 12, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-03-22T19:19:59.236 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 107, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 35 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 111 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-03-22T19:19:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 51, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 1, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 38, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 6654, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 2448475, 2024-03-22T19:19:59.238 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 6654, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 6654, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 2448475, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 1448, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 4260409, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1448, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4260409, 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 1448, 2024-03-22T19:19:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 1448, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1448, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1448, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:59.242 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 38, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 3838, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 38, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3838, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.243 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x561dfe2240a0": { 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3365, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 4225624, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3365, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 182, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4225624, 2024-03-22T19:19:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:19:59.245 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:19:59.245 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:19:59.245 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:19:59.245 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.245 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:19:59.245 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-03-22T19:19:59.471 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:19:59 smithi131 ceph-mon[78559]: pgmap v556: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:19:59.552 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-03-22T19:19:59.552 INFO:tasks.check_counter:Daemon mds.h mds.imported=4 2024-03-22T19:19:59.552 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-03-22T19:19:59.552 DEBUG:tasks.check_counter:Getting stats from k 2024-03-22T19:19:59.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-22T19:19:59.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:19:59 smithi086 ceph-mon[38032]: pgmap v556: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:19:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.b/config 2024-03-22T19:19:59.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:19:59 smithi167 ceph-mon[47728]: pgmap v556: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:00.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:20:00.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:00 smithi131 ceph-mon[78559]: overall HEALTH_OK 2024-03-22T19:20:00.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:20:00.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:00 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:20:00.472 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:00 smithi131 ceph-mon[78559]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:20:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:20:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:00 smithi086 ceph-mon[38032]: overall HEALTH_OK 2024-03-22T19:20:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:20:00.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:00 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:20:00.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:00 smithi086 ceph-mon[38032]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 802, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 779, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 795488, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 288098, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.094370742, 2024-03-22T19:20:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.032623485, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.031219767, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.025225697, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 778, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.158912936, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000204258 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.824 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 597248, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 104608 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 8, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 143, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 4494, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 311337, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-03-22T19:20:00.825 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-03-22T19:20:00.826 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.008398565, 2024-03-22T19:20:00.826 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.005394447, 2024-03-22T19:20:00.826 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.001630190, 2024-03-22T19:20:00.826 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000108839, 2024-03-22T19:20:00.826 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:00.827 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 142, 2024-03-22T19:20:00.827 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.027820825, 2024-03-22T19:20:00.827 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000195921 2024-03-22T19:20:00.827 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.827 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:00.827 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.828 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.828 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.828 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.828 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-22T19:20:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 310336 2024-03-22T19:20:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:20:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1120, 2024-03-22T19:20:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1122, 2024-03-22T19:20:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 4511697, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 348321, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.089880634, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.018986994, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.029275126, 2024-03-22T19:20:00.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.035475195, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1120, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.209048802, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000186650 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:20:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2979, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.075302036, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000025277 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 39, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 39, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-03-22T19:20:00.834 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 39, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-03-22T19:20:00.835 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-03-22T19:20:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-03-22T19:20:00.837 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-03-22T19:20:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-03-22T19:20:00.839 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 3365, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 5, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-03-22T19:20:00.840 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 8419928, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 8418277, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 3365 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-03-22T19:20:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 4, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 78, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 74, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 2, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 16, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 14, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 39, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 113, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 74, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 49588, 2024-03-22T19:20:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 207156 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-03-22T19:20:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-03-22T19:20:00.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.849 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-03-22T19:20:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-03-22T19:20:00.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.852 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-03-22T19:20:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2365, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:20:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-03-22T19:20:00.856 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-03-22T19:20:00.857 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 198247, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 56, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:20:00.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7344, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 155, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 39248, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 184, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.859 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 1713, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 1713, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1713, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 52.231780477, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.030491407 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-03-22T19:20:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1713, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1714 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 1713, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 1713, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 986, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 725, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-03-22T19:20:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-03-22T19:20:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-03-22T19:20:00.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 6, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 6, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-03-22T19:20:00.864 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-03-22T19:20:00.865 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1926, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 5124596, 2024-03-22T19:20:00.866 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1926, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1926, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 5124596, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 1713, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 4225949, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1713, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4225949, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.868 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 1713, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 1713, 2024-03-22T19:20:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1713, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1713, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:20:00.870 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5653ac9ca500": { 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-03-22T19:20:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-03-22T19:20:00.872 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.873 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-03-22T19:20:00.873 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-03-22T19:20:00.873 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T19:20:00.873 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:00 smithi167 ceph-mon[47728]: overall HEALTH_OK 2024-03-22T19:20:00.873 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T19:20:00.873 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:00 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T19:20:00.874 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:00 smithi167 ceph-mon[47728]: from='mgr.14152 ' entity='mgr.x' 2024-03-22T19:20:01.179 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-22T19:20:01.179 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-22T19:20:01.179 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-03-22T19:20:01.179 DEBUG:tasks.check_counter:Getting stats from c 2024-03-22T19:20:01.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-22T19:20:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:01 smithi167 ceph-mon[47728]: pgmap v557: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-22T19:20:01.581 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:20:01.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:01 smithi131 ceph-mon[78559]: pgmap v557: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-22T19:20:01.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:01 smithi086 ceph-mon[38032]: pgmap v557: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-22T19:20:03.057 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-03-22T19:20:03.057 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:20:03.057 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 839, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 841, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4460165, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 260939, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.089220372, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.024819739, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.026250257, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.028990470, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 839, 2024-03-22T19:20:03.058 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.155854972, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000185762 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 988, 2024-03-22T19:20:03.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 963, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2764705, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 344501, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.096200160, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.025877005, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.034965350, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.031891551, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 963, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.238761485, 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000247935 2024-03-22T19:20:03.060 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 599808, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 104032 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 440, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 576, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4305921, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 447810, 2024-03-22T19:20:03.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.048310634, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.017932968, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.012862629, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.012902915, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 574, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.091821722, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000159968 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.062 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 311280 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3445, 2024-03-22T19:20:03.063 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.089532349, 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000025989 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-03-22T19:20:03.064 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-03-22T19:20:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 668, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 668, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 668, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 2, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-03-22T19:20:03.066 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-03-22T19:20:03.067 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:20:03.068 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-03-22T19:20:03.069 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-03-22T19:20:03.070 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 5553, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-03-22T19:20:03.071 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 7, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 14552883, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 14552284, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 5553 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.072 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 18, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 833, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 815, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 13, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 115, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 102, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 668, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 1197, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 529, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 52864, 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207156 2024-03-22T19:20:03.073 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.074 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.075 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.076 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.077 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.078 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-03-22T19:20:03.079 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-03-22T19:20:03.080 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-03-22T19:20:03.082 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.083 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.084 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 2, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-03-22T19:20:03.085 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 4054, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:20:03.086 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:20:03.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 189400, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 51, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-03-22T19:20:03.088 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7344, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 155, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 525994, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2906, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-03-22T19:20:03.089 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 2051, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 2051, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2051, 2024-03-22T19:20:03.090 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 23.752527688, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.011580949 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2051, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2052 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 2051, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 2051, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1221, 2024-03-22T19:20:03.091 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 827, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-03-22T19:20:03.092 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-03-22T19:20:03.093 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 82, 2024-03-22T19:20:03.094 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 5, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 5, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-03-22T19:20:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:20:03.096 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2263, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 11313436, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2263, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2263, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 11313436, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.097 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2051, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 10359005, 2024-03-22T19:20:03.098 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2051, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 10359005, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2051, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2051, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2051, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2051, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.101 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x5627f3d405a0": { 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-03-22T19:20:03.102 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:03.103 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-03-22T19:20:03.412 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-22T19:20:03.412 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-22T19:20:03.412 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-03-22T19:20:03.412 DEBUG:tasks.check_counter:Getting stats from f 2024-03-22T19:20:03.412 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-22T19:20:03.640 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:20:03.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:03 smithi131 ceph-mon[78559]: pgmap v558: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:03.739 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:03 smithi167 ceph-mon[47728]: pgmap v558: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:03.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:03 smithi086 ceph-mon[38032]: pgmap v558: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:04.596 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2273, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2594, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1390014, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 2618977, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 15, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.173959351, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.062550621, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.122756132, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.018636529, 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:04.597 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2591, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.697228405, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000269096 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1813, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000573523, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000316 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 606736, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 112400 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:20:04.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 936, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1036, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 422037, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 783065, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 10, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.089642387, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.032737097, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.051356325, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.011121229, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1037, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.255310648, 2024-03-22T19:20:04.599 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000246201 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1337, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000379060, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000283 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 314608 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 324, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 326, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 170482, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 340447, 2024-03-22T19:20:04.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 25, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.036057908, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.008030617, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.017829205, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.009217759, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 324, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.107655806, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000332271 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 114, 2024-03-22T19:20:04.601 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000024060, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000211 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 446, 2024-03-22T19:20:04.602 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.212372429, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000476171 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 508, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.018032122, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000035496 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 55, 2024-03-22T19:20:04.603 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 54, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 54, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.080117808, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001483663 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 1, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-03-22T19:20:04.604 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 17, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 17, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 14, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 10, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 1, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 1, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 4, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 123, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 90, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 7, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-03-22T19:20:04.605 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 1000, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 1, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 2, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 2, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 18, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 54, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 324, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 13424350788, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-03-22T19:20:04.606 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 54, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 54, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 3, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 225, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:20:04.607 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 1145, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 1012, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 3, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 39 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 3, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 2, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 11, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:20:04.608 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 18, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 18, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 2, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-03-22T19:20:04.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 1265, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 1265, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 2, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-03-22T19:20:04.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 5766080, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 143, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 28.880589944, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.201962167 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 22, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 45, 2024-03-22T19:20:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 23, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 21, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 28, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 7, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 17, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 63, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 46, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 1, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 26, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 25, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 51904, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 198964 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 120, 2024-03-22T19:20:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 2, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 55, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 8, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 7, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.003023858, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001511929 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 32, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.015120097, 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000472503 2024-03-22T19:20:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.617 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001567654, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000783827 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-03-22T19:20:04.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.619 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.621 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-03-22T19:20:04.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 18, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.060406199, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003355899 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 40, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 40, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 557, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.624 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:20:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:20:04.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 284854, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 114, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-03-22T19:20:04.627 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7344, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 155, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 87957, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 375, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-03-22T19:20:04.628 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 788, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 1593774, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 788, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 788, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 130.283434924, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.165334308 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.629 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 788, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 866 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 788, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 55, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 733, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 144, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 31, 2024-03-22T19:20:04.630 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 488, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 54, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-03-22T19:20:04.631 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-03-22T19:20:04.632 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 82, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 92, 2024-03-22T19:20:04.633 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 26 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 59 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-03-22T19:20:04.634 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 51, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 1, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 18, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3529, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1669122, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3529, 2024-03-22T19:20:04.635 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3529, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1669122, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:04.636 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 788, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1595592, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 788, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1595592, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:04.637 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 788, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 788, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 788, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 788, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.638 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 18, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1818, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 18, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1818, 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:04.639 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55f40559e0a0": { 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 1265, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1571776, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 1265, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:04.640 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 143, 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1571776, 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:04.641 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-03-22T19:20:04.970 INFO:tasks.check_counter:Daemon mds.f mds.exported=2 2024-03-22T19:20:04.970 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-03-22T19:20:04.970 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-03-22T19:20:04.970 DEBUG:tasks.check_counter:Getting stats from i 2024-03-22T19:20:04.970 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-22T19:20:05.199 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:20:05.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:05 smithi167 ceph-mon[47728]: pgmap v559: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-03-22T19:20:05.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:05 smithi131 ceph-mon[78559]: pgmap v559: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-03-22T19:20:05.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:05 smithi086 ceph-mon[38032]: pgmap v559: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-03-22T19:20:06.166 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-03-22T19:20:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:20:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 6114, 2024-03-22T19:20:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 7307, 2024-03-22T19:20:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2087651, 2024-03-22T19:20:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 5332154, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.410534663, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.154104320, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.319951384, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.035096336, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7306, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2.078325809, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000284468 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:06.167 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5070, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.002803301, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000552 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2079, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1605, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1286341, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 2963887, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 17, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-03-22T19:20:06.168 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.149703480, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.049148413, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.094103781, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.028719664, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1604, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.336978387, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000210086 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1775, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000293639, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000165 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 615920, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 112304 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1100, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1106, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 527953, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 1116133, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 22, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.083233997, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.030623916, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.055870668, 2024-03-22T19:20:06.170 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.008031872, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1104, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.285292180, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000258416 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1443, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000338617, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000234 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 314176 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.171 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2187, 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.946016440, 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000432563 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.172 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 710, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.023763682, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000033469 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 254, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 247, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 247, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.871366565, 2024-03-22T19:20:06.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003527799 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 7, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 20, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 1, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 17, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 3, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 2, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 12, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 12, 2024-03-22T19:20:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 167, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 1, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 1, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 5, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 390, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 375, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 6, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 2, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 9, 2024-03-22T19:20:06.175 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 129, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 9, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 68, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 158, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 1, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 324, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 13424350788, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 158, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 8, 2024-03-22T19:20:06.176 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 8, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 150, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 10, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 13, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 1062, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 3043, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 2777, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 4, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 58 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.177 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 3, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 2, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 4, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 10, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 28, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 28, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-03-22T19:20:06.178 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 9, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 1, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 3211, 2024-03-22T19:20:06.179 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 3211, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 4, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 4, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 10723880, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-03-22T19:20:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 994, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 310.370978281, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.312244444 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 23, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 158, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 135, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 22, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 38, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 16, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 17, 2024-03-22T19:20:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 239, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 222, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 1, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 174, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 173, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 59432, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 198964 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 274, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 7, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 254, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 16, 2024-03-22T19:20:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 24, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 88, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.068283837, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000775952 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.019789129, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.019789129 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 126, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.124238328, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000986018 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.184 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000211660, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000211660 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-03-22T19:20:06.186 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.187 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.187 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.187 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.187 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-03-22T19:20:06.187 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.187 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-03-22T19:20:06.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001485831, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000495277 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.189 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.620283256, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.310141628 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.190 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-03-22T19:20:06.192 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 26, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.037074524, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001425943 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.193 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-03-22T19:20:06.194 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 62, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 62, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 552, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:20:06.195 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:20:06.196 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:20:06.197 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 314531, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 118, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7344, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 155, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:20:06.198 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 91864, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 352, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1942, 2024-03-22T19:20:06.199 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 6707011, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 27, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1942, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1942, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 638.871336354, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.328975971 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1942, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2032 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1942, 2024-03-22T19:20:06.200 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 160, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 1782, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 2, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 996, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 34, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 1, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 673, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-03-22T19:20:06.201 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 158, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-03-22T19:20:06.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-03-22T19:20:06.203 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 82, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 110, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 28 2024-03-22T19:20:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 274 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 26, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 2, 2024-03-22T19:20:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 28, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 9289, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 3118128, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 9289, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 9289, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 3118128, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1942, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 6709847, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1942, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 6709847, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-03-22T19:20:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1942, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1942, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1942, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1942, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 28, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 2836, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 28, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2836, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x558ad0c940a0": { 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3211, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 6529576, 2024-03-22T19:20:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3211, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 994, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 6529576, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:06.212 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-03-22T19:20:06.527 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-03-22T19:20:06.527 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2024-03-22T19:20:06.528 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-03-22T19:20:06.528 DEBUG:tasks.check_counter:Getting stats from l 2024-03-22T19:20:06.528 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:899aeb57f4ac58b35d5b52546466a4c7cb3bdc46 shell --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-22T19:20:06.755 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/mon.c/config 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 360, 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 362, 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4288239, 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 112024, 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-03-22T19:20:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.033573802, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.008072846, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.012080283, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.009553792, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 360, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.023970151, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000066583 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.690 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 419, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 556, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2442562, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 441015, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.050029788, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.014844649, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.014756535, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.015877016, 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:07.691 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 555, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.114382584, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000206094 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 312256 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-03-22T19:20:07.692 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 795, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 772, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 780048, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 286483, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.092381314, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.027273163, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.032134128, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.028478886, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 771, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.104456601, 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000135481 2024-03-22T19:20:07.693 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 597792, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 105792 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-03-22T19:20:07.694 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2331, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.062524793, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000026823 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-03-22T19:20:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 29, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 29, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 29, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 1, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-03-22T19:20:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-03-22T19:20:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-22T19:20:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-03-22T19:20:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-03-22T19:20:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 3211, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-03-22T19:20:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 4, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 10723880, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 10722229, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 3211 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 4, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 169, 2024-03-22T19:20:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 165, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 2, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 22, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 20, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 29, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 194, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 165, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 48504, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207156 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-03-22T19:20:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-03-22T19:20:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-03-22T19:20:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-03-22T19:20:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-03-22T19:20:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-03-22T19:20:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2350, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-03-22T19:20:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-03-22T19:20:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-03-22T19:20:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 198287, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 57, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-03-22T19:20:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7344, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 155, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 32208, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 144, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-03-22T19:20:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1355, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1355, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1355, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 218.430652408, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.161203433 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1355, 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1356 2024-03-22T19:20:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1355, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 1355, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 782, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 571, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-03-22T19:20:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-03-22T19:20:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-03-22T19:20:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 82, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 6, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 6, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-03-22T19:20:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-03-22T19:20:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 1570, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 7355702, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 1570, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1570, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 7355702, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1355, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 6529581, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1355, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 6529581, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1355, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1355, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1355, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1355, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x564262664500": { 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-03-22T19:20:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-03-22T19:20:07.733 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-03-22T19:20:07.734 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:07 smithi131 ceph-mon[78559]: pgmap v560: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-22T19:20:07.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:07 smithi167 ceph-mon[47728]: pgmap v560: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-22T19:20:07.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:07 smithi086 ceph-mon[38032]: pgmap v560: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-22T19:20:08.053 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-22T19:20:08.053 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-22T19:20:08.053 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-03-22T19:20:08.053 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-22T19:20:08.065 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-22T19:20:08.065 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:20:08.065 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T19:20:08.129 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T19:20:08.130 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T19:20:08.214 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 -- 172.21.15.86:0/1370199359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 msgr2=0x7fe7c41022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/1370199359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c41022c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00099c0 tx=0x7fe7b0009cd0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 -- 172.21.15.86:0/1370199359 shutdown_connections 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/1370199359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4102800 0x7fe7c41090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/1370199359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4100c70 0x7fe7c4101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/1370199359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c41022c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 -- 172.21.15.86:0/1370199359 >> 172.21.15.86:0/1370199359 conn(0x7fe7c40fc1e0 msgr2=0x7fe7c40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 -- 172.21.15.86:0/1370199359 shutdown_connections 2024-03-22T19:20:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.452+0000 7fe7cbe00700 1 -- 172.21.15.86:0/1370199359 wait complete. 2024-03-22T19:20:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 Processor -- start 2024-03-22T19:20:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 -- start start 2024-03-22T19:20:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4100c70 0x7fe7c4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4102800 0x7fe7c40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419aa60 con 0x7fe7c4101e60 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4073c20 con 0x7fe7c4100c70 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.453+0000 7fe7cbe00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4073de0 con 0x7fe7c4102800 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c40731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36356/0 (socket says 172.21.15.86:36356) 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 -- 172.21.15.86:0/801049366 learned_addr learned my addr 172.21.15.86:0/801049366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c9e00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4102800 0x7fe7c40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c9400700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4100c70 0x7fe7c4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 -- 172.21.15.86:0/801049366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4102800 msgr2=0x7fe7c40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4102800 0x7fe7c40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 -- 172.21.15.86:0/801049366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4100c70 msgr2=0x7fe7c4074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4100c70 0x7fe7c4074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 -- 172.21.15.86:0/801049366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00095f0 con 0x7fe7c4101e60 2024-03-22T19:20:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c9e00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4102800 0x7fe7c40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c8a00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c40731a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0003730 tx=0x7fe7b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.454+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b001d070 con 0x7fe7c4101e60 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.455+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7b0005050 con 0x7fe7c4101e60 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.455+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0004b10 con 0x7fe7c4101e60 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.455+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c4074000 con 0x7fe7c4101e60 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.455+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c41a56a0 con 0x7fe7c4101e60 2024-03-22T19:20:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.456+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c4108040 con 0x7fe7c4101e60 2024-03-22T19:20:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.458+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fe7b00051c0 con 0x7fe7c4101e60 2024-03-22T19:20:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.459+0000 7fe7c2000700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7ac0a0b10 0x7fe7ac0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.459+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fe7b00ccd30 con 0x7fe7c4101e60 2024-03-22T19:20:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.459+0000 7fe7c9400700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7ac0a0b10 0x7fe7ac0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.460+0000 7fe7c9400700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7ac0a0b10 0x7fe7ac0a2fd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4101cc0 tx=0x7fe7b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.463+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe7b0096780 con 0x7fe7c4101e60 2024-03-22T19:20:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.706+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7c40679e0 con 0x7fe7c4101e60 2024-03-22T19:20:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.708+0000 7fe7c2000700 1 -- 172.21.15.86:0/801049366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7fe7b0099da0 con 0x7fe7c4101e60 2024-03-22T19:20:08.710 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:20:08.710 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:20:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7ac0a0b10 msgr2=0x7fe7ac0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7ac0a0b10 0x7fe7ac0a2fd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4101cc0 tx=0x7fe7b8009260 comp rx=0 tx=0).stop 2024-03-22T19:20:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 msgr2=0x7fe7c40731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c40731a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0003730 tx=0x7fe7b000f690 comp rx=0 tx=0).stop 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 shutdown_connections 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fe7ac0a0b10 0x7fe7ac0a2fd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c4102800 0x7fe7c40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4100c70 0x7fe7c4074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.710+0000 7fe7cbe00700 1 --2- 172.21.15.86:0/801049366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4101e60 0x7fe7c40731a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.711+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 >> 172.21.15.86:0/801049366 conn(0x7fe7c40fc1e0 msgr2=0x7fe7c4105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.711+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 shutdown_connections 2024-03-22T19:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.711+0000 7fe7cbe00700 1 -- 172.21.15.86:0/801049366 wait complete. 2024-03-22T19:20:08.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T19:05:46.700016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T19:09:12.040857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T19:09:13.060199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T19:20:08.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 -- 172.21.15.86:0/372148610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 msgr2=0x7fa75c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 --2- 172.21.15.86:0/372148610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c0731e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009990 tx=0x7fa74c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 -- 172.21.15.86:0/372148610 shutdown_connections 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 --2- 172.21.15.86:0/372148610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 0x7fa75c10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 --2- 172.21.15.86:0/372148610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c0731e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 --2- 172.21.15.86:0/372148610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa75c0737b0 0x7fa75c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 -- 172.21.15.86:0/372148610 >> 172.21.15.86:0/372148610 conn(0x7fa75c0fc2c0 msgr2=0x7fa75c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 -- 172.21.15.86:0/372148610 shutdown_connections 2024-03-22T19:20:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.927+0000 7fa760c00700 1 -- 172.21.15.86:0/372148610 wait complete. 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 Processor -- start 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 -- start start 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa75c0737b0 0x7fa75c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 0x7fa75c1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a0a40 con 0x7fa75c074d90 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a0b80 con 0x7fa75c108c20 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.928+0000 7fa760c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a0d10 con 0x7fa75c0737b0 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa75c0737b0 0x7fa75c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36376/0 (socket says 172.21.15.86:36376) 2024-03-22T19:20:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 -- 172.21.15.86:0/3833278185 learned_addr learned my addr 172.21.15.86:0/3833278185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa75a800700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 0x7fa75c1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 -- 172.21.15.86:0/3833278185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa75c0737b0 msgr2=0x7fa75c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa75c0737b0 0x7fa75c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 -- 172.21.15.86:0/3833278185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 msgr2=0x7fa75c1a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 0x7fa75c1a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 -- 172.21.15.86:0/3833278185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0095f0 con 0x7fa75c074d90 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa75a800700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 0x7fa75c1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa759400700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c199450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa74800d980 tx=0x7fa74800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748004220 con 0x7fa75c074d90 2024-03-22T19:20:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa748007640 con 0x7fa75c074d90 2024-03-22T19:20:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.929+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c1a0ff0 con 0x7fa75c074d90 2024-03-22T19:20:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.930+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748005610 con 0x7fa75c074d90 2024-03-22T19:20:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.930+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1a14c0 con 0x7fa75c074d90 2024-03-22T19:20:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.931+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c107a30 con 0x7fa75c074d90 2024-03-22T19:20:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.932+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fa748004750 con 0x7fa75c074d90 2024-03-22T19:20:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.934+0000 7fa752a00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7400a0b10 0x7fa7400a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.934+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fa7480cbe60 con 0x7fa75c074d90 2024-03-22T19:20:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.934+0000 7fa759e00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7400a0b10 0x7fa7400a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.935+0000 7fa759e00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7400a0b10 0x7fa7400a2fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b2e0 tx=0x7fa74c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:08.938+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa748095820 con 0x7fa75c074d90 2024-03-22T19:20:09.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.142+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa75c0619c0 con 0x7fa75c074d90 2024-03-22T19:20:09.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.143+0000 7fa752a00700 1 -- 172.21.15.86:0/3833278185 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa748016020 con 0x7fa75c074d90 2024-03-22T19:20:09.144 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T19:20:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7400a0b10 msgr2=0x7fa7400a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7400a0b10 0x7fa7400a2fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b2e0 tx=0x7fa74c005fd0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 msgr2=0x7fa75c199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c199450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa74800d980 tx=0x7fa74800dc90 comp rx=0 tx=0).stop 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 shutdown_connections 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fa7400a0b10 0x7fa7400a2fd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75c108c20 0x7fa75c1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa75c0737b0 0x7fa75c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 --2- 172.21.15.86:0/3833278185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75c074d90 0x7fa75c199450 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.145+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 >> 172.21.15.86:0/3833278185 conn(0x7fa75c0fc2c0 msgr2=0x7fa75c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.146+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 shutdown_connections 2024-03-22T19:20:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.146+0000 7fa760c00700 1 -- 172.21.15.86:0/3833278185 wait complete. 2024-03-22T19:20:09.166 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 -- 172.21.15.86:0/467208888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1090101f30 msgr2=0x7f1090102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 --2- 172.21.15.86:0/467208888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1090101f30 0x7f1090102390 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f107c0099c0 tx=0x7f107c009cd0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 -- 172.21.15.86:0/467208888 shutdown_connections 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 --2- 172.21.15.86:0/467208888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10901028d0 0x7f1090109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 --2- 172.21.15.86:0/467208888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1090100d40 0x7f1090101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 --2- 172.21.15.86:0/467208888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1090101f30 0x7f1090102390 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.360+0000 7f1098200700 1 -- 172.21.15.86:0/467208888 >> 172.21.15.86:0/467208888 conn(0x7f10900fc2b0 msgr2=0x7f10900fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.361+0000 7f1098200700 1 -- 172.21.15.86:0/467208888 shutdown_connections 2024-03-22T19:20:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.361+0000 7f1098200700 1 -- 172.21.15.86:0/467208888 wait complete. 2024-03-22T19:20:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 Processor -- start 2024-03-22T19:20:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 -- start start 2024-03-22T19:20:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1090100d40 0x7f1090074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1090101f30 0x7f10900731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 0x7f10900736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090073e80 con 0x7f10901028d0 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090073ff0 con 0x7f1090100d40 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1098200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10901a0c90 con 0x7f1090101f30 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1096200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 0x7f10900736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1096200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 0x7f10900736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36400/0 (socket says 172.21.15.86:36400) 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1096200700 1 -- 172.21.15.86:0/1179344965 learned_addr learned my addr 172.21.15.86:0/1179344965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1094e00700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1090101f30 0x7f10900731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.362+0000 7f1095800700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1090100d40 0x7f1090074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1096200700 1 -- 172.21.15.86:0/1179344965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1090101f30 msgr2=0x7f10900731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1096200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1090101f30 0x7f10900731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1096200700 1 -- 172.21.15.86:0/1179344965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1090100d40 msgr2=0x7f1090074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1096200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1090100d40 0x7f1090074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1096200700 1 -- 172.21.15.86:0/1179344965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0095f0 con 0x7f10901028d0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1096200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 0x7f10900736e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f108800ef30 tx=0x7f108800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088018070 con 0x7f10901028d0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108800f040 con 0x7f10901028d0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10901a0ec0 con 0x7f10901028d0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800cca0 con 0x7f10901028d0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.363+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10901a1340 con 0x7f10901028d0 2024-03-22T19:20:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.365+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10901081e0 con 0x7f10901028d0 2024-03-22T19:20:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.366+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f1088003a00 con 0x7f10901028d0 2024-03-22T19:20:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.368+0000 7f108e000700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10780a0a90 0x7f10780a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.368+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f10880cbb70 con 0x7f10901028d0 2024-03-22T19:20:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.368+0000 7f1095800700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10780a0a90 0x7f10780a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.368+0000 7f1095800700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10780a0a90 0x7f10780a2f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1090101d90 tx=0x7f108400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:09.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.371+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f10880955c0 con 0x7f10901028d0 2024-03-22T19:20:09.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.580+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10900619c0 con 0x7f10901028d0 2024-03-22T19:20:09.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.580+0000 7f108e000700 1 -- 172.21.15.86:0/1179344965 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1088098be0 con 0x7f10901028d0 2024-03-22T19:20:09.582 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10780a0a90 msgr2=0x7f10780a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10780a0a90 0x7f10780a2f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1090101d90 tx=0x7f108400b410 comp rx=0 tx=0).stop 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 msgr2=0x7f10900736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 0x7f10900736e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f108800ef30 tx=0x7f108800c3a0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 shutdown_connections 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f10780a0a90 0x7f10780a2f50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10901028d0 0x7f10900736e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1090100d40 0x7f1090074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 --2- 172.21.15.86:0/1179344965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1090101f30 0x7f10900731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 >> 172.21.15.86:0/1179344965 conn(0x7f10900fc2b0 msgr2=0x7f1090105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 shutdown_connections 2024-03-22T19:20:09.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.583+0000 7f1098200700 1 -- 172.21.15.86:0/1179344965 wait complete. 2024-03-22T19:20:09.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T19:20:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:09 smithi131 ceph-mon[78559]: pgmap v561: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:09 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/801049366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:20:09.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:09 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3833278185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:20:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:09 smithi086 ceph-mon[38032]: pgmap v561: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:09 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/801049366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:20:09.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:09 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3833278185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:20:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 -- 172.21.15.86:0/3223119033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 msgr2=0x7fca8810b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 --2- 172.21.15.86:0/3223119033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca8810b0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fca8000b0a0 tx=0x7fca8000b3b0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 -- 172.21.15.86:0/3223119033 shutdown_connections 2024-03-22T19:20:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 --2- 172.21.15.86:0/3223119033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca8810b0a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 --2- 172.21.15.86:0/3223119033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca88074df0 0x7fca88073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 --2- 172.21.15.86:0/3223119033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca88073810 0x7fca88073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 -- 172.21.15.86:0/3223119033 >> 172.21.15.86:0/3223119033 conn(0x7fca880fc2c0 msgr2=0x7fca880fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 -- 172.21.15.86:0/3223119033 shutdown_connections 2024-03-22T19:20:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.799+0000 7fca8e800700 1 -- 172.21.15.86:0/3223119033 wait complete. 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.800+0000 7fca8e800700 1 Processor -- start 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.800+0000 7fca8e800700 1 -- start start 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca8e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca88073810 0x7fca88198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca8e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca88074df0 0x7fca88199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca881a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca8e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca881a0b20 con 0x7fca88108c10 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca8e800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca881a0c60 con 0x7fca88074df0 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca8e800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca881a0df0 con 0x7fca88073810 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca881a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca88073810 0x7fca88198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca86a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca88074df0 0x7fca88199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca881a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36414/0 (socket says 172.21.15.86:36414) 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 -- 172.21.15.86:0/1866758152 learned_addr learned my addr 172.21.15.86:0/1866758152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 -- 172.21.15.86:0/1866758152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca88073810 msgr2=0x7fca88198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca88073810 0x7fca88198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 -- 172.21.15.86:0/1866758152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca88074df0 msgr2=0x7fca88199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca88074df0 0x7fca88199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.801+0000 7fca87e00700 1 -- 172.21.15.86:0/1866758152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80009d00 con 0x7fca88108c10 2024-03-22T19:20:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.802+0000 7fca87e00700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca881a0550 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fca80007a60 tx=0x7fca80007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.802+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000e040 con 0x7fca88108c10 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.802+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca80004c50 con 0x7fca88108c10 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.802+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca881a1070 con 0x7fca88108c10 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.802+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8001b6c0 con 0x7fca88108c10 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.802+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca881a1490 con 0x7fca88108c10 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.803+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca881079d0 con 0x7fca88108c10 2024-03-22T19:20:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.805+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fca8001d470 con 0x7fca88108c10 2024-03-22T19:20:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.806+0000 7fca7fe00700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fca6c0a0b10 0x7fca6c0a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.806+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fca800cd250 con 0x7fca88108c10 2024-03-22T19:20:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.806+0000 7fca87400700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fca6c0a0b10 0x7fca6c0a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.807+0000 7fca87400700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fca6c0a0b10 0x7fca6c0a2fd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca78009c60 tx=0x7fca78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:09.810+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fca80096ca0 con 0x7fca88108c10 2024-03-22T19:20:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:09 smithi167 ceph-mon[47728]: pgmap v561: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-22T19:20:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:09 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/801049366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:20:09.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:09 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3833278185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.017+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca8802cee0 con 0x7fca88108c10 2024-03-22T19:20:10.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.018+0000 7fca7fe00700 1 -- 172.21.15.86:0/1866758152 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca8009a2c0 con 0x7fca88108c10 2024-03-22T19:20:10.019 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T19:20:10.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fca6c0a0b10 msgr2=0x7fca6c0a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fca6c0a0b10 0x7fca6c0a2fd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca78009c60 tx=0x7fca78009260 comp rx=0 tx=0).stop 2024-03-22T19:20:10.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 msgr2=0x7fca881a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca881a0550 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fca80007a60 tx=0x7fca80007930 comp rx=0 tx=0).stop 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 shutdown_connections 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fca6c0a0b10 0x7fca6c0a2fd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca88108c10 0x7fca881a0550 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca88073810 0x7fca88198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 --2- 172.21.15.86:0/1866758152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca88074df0 0x7fca88199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.020+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 >> 172.21.15.86:0/1866758152 conn(0x7fca880fc2c0 msgr2=0x7fca88104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.021+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 shutdown_connections 2024-03-22T19:20:10.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.021+0000 7fca8e800700 1 -- 172.21.15.86:0/1866758152 wait complete. 2024-03-22T19:20:10.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T19:20:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 -- 172.21.15.86:0/2201266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc101f30 msgr2=0x7f6afc102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 --2- 172.21.15.86:0/2201266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc101f30 0x7f6afc102390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009920 tx=0x7f6ae8009c30 comp rx=0 tx=0).stop 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 -- 172.21.15.86:0/2201266324 shutdown_connections 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 --2- 172.21.15.86:0/2201266324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc1028d0 0x7f6afc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 --2- 172.21.15.86:0/2201266324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc100d40 0x7f6afc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 --2- 172.21.15.86:0/2201266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc101f30 0x7f6afc102390 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.235+0000 7f6b04200700 1 -- 172.21.15.86:0/2201266324 >> 172.21.15.86:0/2201266324 conn(0x7f6afc0fc2b0 msgr2=0x7f6afc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.236+0000 7f6b04200700 1 -- 172.21.15.86:0/2201266324 shutdown_connections 2024-03-22T19:20:10.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.236+0000 7f6b04200700 1 -- 172.21.15.86:0/2201266324 wait complete. 2024-03-22T19:20:10.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 Processor -- start 2024-03-22T19:20:10.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 -- start start 2024-03-22T19:20:10.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc100d40 0x7f6afc196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:10.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 0x7f6afc1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:10.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc1028d0 0x7f6afc19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:10.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc19e890 con 0x7f6afc1028d0 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc19ea00 con 0x7f6afc100d40 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.237+0000 7f6b04200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc19ebc0 con 0x7f6afc101f30 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b01800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc100d40 0x7f6afc196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 0x7f6afc1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b02200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc1028d0 0x7f6afc19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b01800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc100d40 0x7f6afc196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44118/0 (socket says 172.21.15.86:44118) 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b01800700 1 -- 172.21.15.86:0/3685684183 learned_addr learned my addr 172.21.15.86:0/3685684183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 0x7f6afc1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51642/0 (socket says 172.21.15.86:51642) 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 -- 172.21.15.86:0/3685684183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc100d40 msgr2=0x7f6afc196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc100d40 0x7f6afc196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 -- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc1028d0 msgr2=0x7f6afc19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc1028d0 0x7f6afc19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 -- 172.21.15.86:0/3685684183 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6afc101f30 2024-03-22T19:20:10.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b02200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc1028d0 0x7f6afc19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:10.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6b00e00700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 0x7f6afc1972e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009230 tx=0x7f6ae800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:10.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.238+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae801c070 con 0x7f6afc101f30 2024-03-22T19:20:10.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.239+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae8009520 con 0x7f6afc101f30 2024-03-22T19:20:10.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.239+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae800f7d0 con 0x7f6afc101f30 2024-03-22T19:20:10.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.239+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8004910 con 0x7f6afc101f30 2024-03-22T19:20:10.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.239+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc19f080 con 0x7f6afc101f30 2024-03-22T19:20:10.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.240+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc108200 con 0x7f6afc101f30 2024-03-22T19:20:10.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.242+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f6ae8020070 con 0x7f6afc101f30 2024-03-22T19:20:10.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.243+0000 7f6afa000700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6ae40a0a90 0x7f6ae40a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:10.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.243+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f6ae80cd100 con 0x7f6afc101f30 2024-03-22T19:20:10.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.243+0000 7f6b01800700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6ae40a0a90 0x7f6ae40a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:10.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.244+0000 7f6b01800700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6ae40a0a90 0x7f6ae40a2f50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6afc198220 tx=0x7f6af0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.246+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6ae8096b50 con 0x7f6afc101f30 2024-03-22T19:20:10.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.508+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6afc067ac0 con 0x7f6afc101f30 2024-03-22T19:20:10.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.509+0000 7f6afa000700 1 -- 172.21.15.86:0/3685684183 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) v1 ==== 81+0+590 (secure 0 0 0) 0x7f6ae809a170 con 0x7f6afc101f30 2024-03-22T19:20:10.510 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1950872310 2024-03-23T19:03:50.084047+0000 2024-03-22T19:20:10.510 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3835111951 2024-03-23T19:03:25.120388+0000 2024-03-22T19:20:10.510 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2748771516 2024-03-23T19:03:50.084047+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2290148199 2024-03-23T19:03:25.120388+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2748771516 2024-03-23T19:03:50.084047+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2112495366 2024-03-23T19:03:25.120388+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/764935385 2024-03-23T19:03:50.084047+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/583320025 2024-03-23T19:03:25.120388+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1207258255 2024-03-23T19:03:50.084047+0000 2024-03-22T19:20:10.511 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2290148199 2024-03-23T19:03:25.120388+0000 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6ae40a0a90 msgr2=0x7f6ae40a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6ae40a0a90 0x7f6ae40a2f50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6afc198220 tx=0x7f6af0009310 comp rx=0 tx=0).stop 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 msgr2=0x7f6afc1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 0x7f6afc1972e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009230 tx=0x7f6ae800b790 comp rx=0 tx=0).stop 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 shutdown_connections 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f6ae40a0a90 0x7f6ae40a2f50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc1028d0 0x7f6afc19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc100d40 0x7f6afc196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 --2- 172.21.15.86:0/3685684183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6afc101f30 0x7f6afc1972e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:10.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 >> 172.21.15.86:0/3685684183 conn(0x7f6afc0fc2b0 msgr2=0x7f6afc105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:10.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 shutdown_connections 2024-03-22T19:20:10.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:10.511+0000 7f6b04200700 1 -- 172.21.15.86:0/3685684183 wait complete. 2024-03-22T19:20:10.514 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-03-22T19:20:10.533 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T19:20:10.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:20:10.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T19:20:10.637 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T19:20:10.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:20:10.637 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T19:20:10.727 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:10 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1179344965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.727 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:10 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/1866758152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:20:10.745 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T19:20:10.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1179344965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:10 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/1866758152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:10 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1179344965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:10 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/1866758152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-22T19:20:10.873 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T19:20:10.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:20:10.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T19:20:11.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T19:20:11.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-22T19:20:11.171 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-22T19:20:11.182 INFO:tasks.cephadm:Teardown begin 2024-03-22T19:20:11.182 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T19:20:11.182 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:20:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 -- 172.21.15.86:0/1932058410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c1027e0 msgr2=0x7ff91c109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 --2- 172.21.15.86:0/1932058410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c1027e0 0x7ff91c109090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff918009fc0 tx=0x7ff91800b330 comp rx=0 tx=0).stop 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 -- 172.21.15.86:0/1932058410 shutdown_connections 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 --2- 172.21.15.86:0/1932058410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c1027e0 0x7ff91c109090 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 --2- 172.21.15.86:0/1932058410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 --2- 172.21.15.86:0/1932058410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c101e40 0x7ff91c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 -- 172.21.15.86:0/1932058410 >> 172.21.15.86:0/1932058410 conn(0x7ff91c0fc1c0 msgr2=0x7ff91c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.377+0000 7ff924800700 1 -- 172.21.15.86:0/1932058410 shutdown_connections 2024-03-22T19:20:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.378+0000 7ff924800700 1 -- 172.21.15.86:0/1932058410 wait complete. 2024-03-22T19:20:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.378+0000 7ff924800700 1 Processor -- start 2024-03-22T19:20:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 -- start start 2024-03-22T19:20:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c101e40 0x7ff91c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1027e0 0x7ff91c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1a09e0 con 0x7ff91c1027e0 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1a0b20 con 0x7ff91c101e40 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff924800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1a0cb0 con 0x7ff91c100c50 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff921e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff921e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51658/0 (socket says 172.21.15.86:51658) 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff921e00700 1 -- 172.21.15.86:0/211390465 learned_addr learned my addr 172.21.15.86:0/211390465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff921400700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c101e40 0x7ff91c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.379+0000 7ff922800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1027e0 0x7ff91c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff921e00700 1 -- 172.21.15.86:0/211390465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c101e40 msgr2=0x7ff91c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff921e00700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c101e40 0x7ff91c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff921e00700 1 -- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1027e0 msgr2=0x7ff91c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff921e00700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1027e0 0x7ff91c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff921e00700 1 -- 172.21.15.86:0/211390465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009c40 con 0x7ff91c100c50 2024-03-22T19:20:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff921e00700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c198f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e970 tx=0x7ff91000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910009790 con 0x7ff91c100c50 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91000c7c0 con 0x7ff91c100c50 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c1a0f90 con 0x7ff91c100c50 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910003c20 con 0x7ff91c100c50 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.380+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c1a13b0 con 0x7ff91c100c50 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.381+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c067a50 con 0x7ff91c100c50 2024-03-22T19:20:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.383+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff91000c930 con 0x7ff91c100c50 2024-03-22T19:20:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.384+0000 7ff916a00700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9040a0a90 0x7ff9040a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.384+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7ff910014070 con 0x7ff91c100c50 2024-03-22T19:20:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.384+0000 7ff921400700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9040a0a90 0x7ff9040a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.385+0000 7ff921400700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9040a0a90 0x7ff9040a2f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:11.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.386+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff910095ce0 con 0x7ff91c100c50 2024-03-22T19:20:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.641+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff91c1a1600 con 0x7ff91c100c50 2024-03-22T19:20:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.642+0000 7ff916a00700 1 -- 172.21.15.86:0/211390465 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10931 (secure 0 0 0) 0x7ff910099300 con 0x7ff91c100c50 2024-03-22T19:20:11.644 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:20:11.644 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:19:16.508755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9040a0a90 msgr2=0x7ff9040a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9040a0a90 0x7ff9040a2f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c009310 comp rx=0 tx=0).stop 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 msgr2=0x7ff91c198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c198f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e970 tx=0x7ff91000ec80 comp rx=0 tx=0).stop 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 shutdown_connections 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9040a0a90 0x7ff9040a2f50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1027e0 0x7ff91c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c100c50 0x7ff91c198f00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 --2- 172.21.15.86:0/211390465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff91c101e40 0x7ff91c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 >> 172.21.15.86:0/211390465 conn(0x7ff91c0fc1c0 msgr2=0x7ff91c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 shutdown_connections 2024-03-22T19:20:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.645+0000 7ff924800700 1 -- 172.21.15.86:0/211390465 wait complete. 2024-03-22T19:20:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-03-22T19:20:11.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:11 smithi086 ceph-mon[38032]: pgmap v562: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-22T19:20:11.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:11 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/3685684183' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T19:20:11.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T19:20:11.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:11 smithi131 ceph-mon[78559]: pgmap v562: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-22T19:20:11.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:11 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/3685684183' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T19:20:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:11 smithi167 ceph-mon[47728]: pgmap v562: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-22T19:20:11.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:11 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/3685684183' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T19:20:11.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 -- 172.21.15.86:0/4037882335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac101e40 msgr2=0x7fb2ac1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/4037882335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1022a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb2980099c0 tx=0x7fb298009cd0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 -- 172.21.15.86:0/4037882335 shutdown_connections 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/4037882335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac1027e0 0x7fb2ac109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/4037882335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/4037882335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1022a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.861+0000 7fb2b3c00700 1 -- 172.21.15.86:0/4037882335 >> 172.21.15.86:0/4037882335 conn(0x7fb2ac0fc1c0 msgr2=0x7fb2ac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.862+0000 7fb2b3c00700 1 -- 172.21.15.86:0/4037882335 shutdown_connections 2024-03-22T19:20:11.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.862+0000 7fb2b3c00700 1 -- 172.21.15.86:0/4037882335 wait complete. 2024-03-22T19:20:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.862+0000 7fb2b3c00700 1 Processor -- start 2024-03-22T19:20:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 -- start start 2024-03-22T19:20:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac1027e0 0x7fb2ac1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac1a08c0 con 0x7fb2ac1027e0 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac1a0a00 con 0x7fb2ac101e40 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b3c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac1a0b90 con 0x7fb2ac100c50 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b1200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b1200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.86:51672/0 (socket says 172.21.15.86:51672) 2024-03-22T19:20:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b1200700 1 -- 172.21.15.86:0/884010684 learned_addr learned my addr 172.21.15.86:0/884010684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2b1c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac1027e0 0x7fb2ac1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.863+0000 7fb2abe00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1200700 1 -- 172.21.15.86:0/884010684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac101e40 msgr2=0x7fb2ac1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1200700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1200700 1 -- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac1027e0 msgr2=0x7fb2ac1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1200700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac1027e0 0x7fb2ac1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1200700 1 -- 172.21.15.86:0/884010684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980095f0 con 0x7fb2ac100c50 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2abe00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac1027e0 0x7fb2ac1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b1200700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac198d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e950 tx=0x7fb2a000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0009770 con 0x7fb2ac100c50 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2a000eea0 con 0x7fb2ac100c50 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac1a0e70 con 0x7fb2ac100c50 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0003cf0 con 0x7fb2ac100c50 2024-03-22T19:20:11.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.864+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac1a1240 con 0x7fb2ac100c50 2024-03-22T19:20:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.865+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac067a50 con 0x7fb2ac100c50 2024-03-22T19:20:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.867+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7fb2a000ccd0 con 0x7fb2ac100c50 2024-03-22T19:20:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.868+0000 7fb2a9600700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb2940a0a90 0x7fb2940a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.868+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fb2a0014070 con 0x7fb2ac100c50 2024-03-22T19:20:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.868+0000 7fb2abe00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb2940a0a90 0x7fb2940a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.869+0000 7fb2abe00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb2940a0a90 0x7fb2940a2f50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b580 tx=0x7fb29801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:11.870+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb2a0095c40 con 0x7fb2ac100c50 2024-03-22T19:20:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.124+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2ac0679e0 con 0x7fb2ac100c50 2024-03-22T19:20:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.126+0000 7fb2a9600700 1 -- 172.21.15.86:0/884010684 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10931 (secure 0 0 0) 0x7fb2a0099260 con 0x7fb2ac100c50 2024-03-22T19:20:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:20:12.127 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1672168335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1672168335},{"type":"v1","addr":"172.21.15.86:6835","nonce":1672168335}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3289005201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3289005201},{"type":"v1","addr":"172.21.15.86:6837","nonce":3289005201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T19:09:13.618776+0000","modified":"2024-03-22T19:19:16.508755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24550,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3022543840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3022543840},{"type":"v1","addr":"172.21.15.86:6839","nonce":3022543840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24470":{"gid":24470,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/2878260168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2878260168},{"type":"v1","addr":"172.21.15.131:6833","nonce":2878260168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/700195904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":700195904},{"type":"v1","addr":"172.21.15.167:6833","nonce":700195904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/4106390239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4106390239},{"type":"v1","addr":"172.21.15.131:6835","nonce":4106390239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1695283976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1695283976},{"type":"v1","addr":"172.21.15.167:6837","nonce":1695283976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6835/2161889252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2161889252},{"type":"v1","addr":"172.21.15.167:6835","nonce":2161889252}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6837/3746834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3746834114},{"type":"v1","addr":"172.21.15.131:6837","nonce":3746834114}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3747705607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3747705607},{"type":"v1","addr":"172.21.15.167:6839","nonce":3747705607}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6841/3747210669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3747210669},{"type":"v1","addr":"172.21.15.86:6841","nonce":3747210669}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6839/1545428523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1545428523},{"type":"v1","addr":"172.21.15.131:6839","nonce":1545428523}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-22T19:20:12.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.127+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb2940a0a90 msgr2=0x7fb2940a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.127+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb2940a0a90 0x7fb2940a2f50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b580 tx=0x7fb29801a040 comp rx=0 tx=0).stop 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.127+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 msgr2=0x7fb2ac198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.127+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac198d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e950 tx=0x7fb2a000ec60 comp rx=0 tx=0).stop 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 shutdown_connections 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7fb2940a0a90 0x7fb2940a2f50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac1027e0 0x7fb2ac1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2ac100c50 0x7fb2ac198d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 --2- 172.21.15.86:0/884010684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2ac101e40 0x7fb2ac1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 >> 172.21.15.86:0/884010684 conn(0x7fb2ac0fc1c0 msgr2=0x7fb2ac072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 shutdown_connections 2024-03-22T19:20:12.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.128+0000 7fb2b3c00700 1 -- 172.21.15.86:0/884010684 wait complete. 2024-03-22T19:20:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-03-22T19:20:12.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T19:20:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 -- 172.21.15.86:0/1104873684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a301027e0 msgr2=0x7f7a30109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 --2- 172.21.15.86:0/1104873684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a301027e0 0x7f7a30109090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b0d0 tx=0x7f7a2800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 -- 172.21.15.86:0/1104873684 shutdown_connections 2024-03-22T19:20:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 --2- 172.21.15.86:0/1104873684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a301027e0 0x7f7a30109090 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 --2- 172.21.15.86:0/1104873684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 0x7f7a30101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 --2- 172.21.15.86:0/1104873684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a30101e40 0x7f7a301022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 -- 172.21.15.86:0/1104873684 >> 172.21.15.86:0/1104873684 conn(0x7f7a300fc1c0 msgr2=0x7f7a300fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 -- 172.21.15.86:0/1104873684 shutdown_connections 2024-03-22T19:20:12.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.340+0000 7f7a36a00700 1 -- 172.21.15.86:0/1104873684 wait complete. 2024-03-22T19:20:12.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 Processor -- start 2024-03-22T19:20:12.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 -- start start 2024-03-22T19:20:12.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 0x7f7a30196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 0x7f7a30197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a301027e0 0x7f7a3019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019e800 con 0x7f7a30101e40 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.341+0000 7f7a36a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019e970 con 0x7f7a301027e0 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a36a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019eb30 con 0x7f7a30100c50 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 0x7f7a30197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 0x7f7a30197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36490/0 (socket says 172.21.15.86:36490) 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 -- 172.21.15.86:0/918221296 learned_addr learned my addr 172.21.15.86:0/918221296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a34a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a301027e0 0x7f7a3019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a2fe00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 0x7f7a30196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 -- 172.21.15.86:0/918221296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 msgr2=0x7f7a30196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 0x7f7a30196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 -- 172.21.15.86:0/918221296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a301027e0 msgr2=0x7f7a3019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a301027e0 0x7f7a3019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 -- 172.21.15.86:0/918221296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28009d00 con 0x7f7a30101e40 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a2fe00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 0x7f7a30196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a34a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a301027e0 0x7f7a3019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.342+0000 7f7a27e00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 0x7f7a30197220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800e9a0 tx=0x7f7a1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.343+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18009790 con 0x7f7a30101e40 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.343+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3019ee10 con 0x7f7a30101e40 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.343+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a1800cbd0 con 0x7f7a30101e40 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.343+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18018680 con 0x7f7a30101e40 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.343+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3019f290 con 0x7f7a30101e40 2024-03-22T19:20:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.344+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a301080a0 con 0x7f7a30101e40 2024-03-22T19:20:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.346+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f7a1800cd40 con 0x7f7a30101e40 2024-03-22T19:20:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.348+0000 7f7a2d600700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a140a0a90 0x7f7a140a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.348+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f7a18014070 con 0x7f7a30101e40 2024-03-22T19:20:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.348+0000 7f7a2fe00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a140a0a90 0x7f7a140a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.348+0000 7f7a2fe00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a140a0a90 0x7f7a140a2f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7a300fe5c0 tx=0x7f7a20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.349+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a18095f50 con 0x7f7a30101e40 2024-03-22T19:20:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.595+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a300679e0 con 0x7f7a30101e40 2024-03-22T19:20:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.598+0000 7f7a2d600700 1 -- 172.21.15.86:0/918221296 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22478 (secure 0 0 0) 0x7f7a18099570 con 0x7f7a30101e40 2024-03-22T19:20:12.599 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T19:20:12.599 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"b22c63ac-e87e-11ee-95cd-87774f69a715","created":"2024-03-22T19:02:55.168034+0000","modified":"2024-03-22T19:09:51.217885+0000","last_up_change":"2024-03-22T19:08:34.798118+0000","last_in_change":"2024-03-22T19:08:18.758442+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T19:05:46.700016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T19:09:12.040857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T19:09:13.060199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edf49b32-3333-4f40-bf4d-fb6ac26e3890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6803","nonce":2783447556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6805","nonce":2783447556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6809","nonce":2783447556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2783447556},{"type":"v1","addr":"172.21.15.86:6807","nonce":2783447556}]},"public_addr":"172.21.15.86:6803/2783447556","cluster_addr":"172.21.15.86:6805/2783447556","heartbeat_back_addr":"172.21.15.86:6809/2783447556","heartbeat_front_addr":"172.21.15.86:6807/2783447556","state":["exists","up"]},{"osd":1,"uuid":"3dc78998-5904-4af2-97fd-880603deeb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6801","nonce":3559403915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6803","nonce":3559403915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6807","nonce":3559403915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3559403915},{"type":"v1","addr":"172.21.15.131:6805","nonce":3559403915}]},"public_addr":"172.21.15.131:6801/3559403915","cluster_addr":"172.21.15.131:6803/3559403915","heartbeat_back_addr":"172.21.15.131:6807/3559403915","heartbeat_front_addr":"172.21.15.131:6805/3559403915","state":["exists","up"]},{"osd":2,"uuid":"84f39745-e2e8-4b26-8041-28d4e8914192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6801","nonce":2732206899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6803","nonce":2732206899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6807","nonce":2732206899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2732206899},{"type":"v1","addr":"172.21.15.167:6805","nonce":2732206899}]},"public_addr":"172.21.15.167:6801/2732206899","cluster_addr":"172.21.15.167:6803/2732206899","heartbeat_back_addr":"172.21.15.167:6807/2732206899","heartbeat_front_addr":"172.21.15.167:6805/2732206899","state":["exists","up"]},{"osd":3,"uuid":"4c78c338-9e45-43c1-abed-bed5550c0582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6811","nonce":1218616366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6813","nonce":1218616366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6817","nonce":1218616366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1218616366},{"type":"v1","addr":"172.21.15.86:6815","nonce":1218616366}]},"public_addr":"172.21.15.86:6811/1218616366","cluster_addr":"172.21.15.86:6813/1218616366","heartbeat_back_addr":"172.21.15.86:6817/1218616366","heartbeat_front_addr":"172.21.15.86:6815/1218616366","state":["exists","up"]},{"osd":4,"uuid":"58f48274-3606-4e03-bd1c-1f58c186882d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6809","nonce":2968759131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6811","nonce":2968759131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6815","nonce":2968759131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2968759131},{"type":"v1","addr":"172.21.15.131:6813","nonce":2968759131}]},"public_addr":"172.21.15.131:6809/2968759131","cluster_addr":"172.21.15.131:6811/2968759131","heartbeat_back_addr":"172.21.15.131:6815/2968759131","heartbeat_front_addr":"172.21.15.131:6813/2968759131","state":["exists","up"]},{"osd":5,"uuid":"4140705c-b24d-488d-8db0-7f6093013c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6809","nonce":634422158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6811","nonce":634422158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6815","nonce":634422158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":634422158},{"type":"v1","addr":"172.21.15.167:6813","nonce":634422158}]},"public_addr":"172.21.15.167:6809/634422158","cluster_addr":"172.21.15.167:6811/634422158","heartbeat_back_addr":"172.21.15.167:6815/634422158","heartbeat_front_addr":"172.21.15.167:6813/634422158","state":["exists","up"]},{"osd":6,"uuid":"e9f54546-8e74-4eb8-9489-9b5155f0501e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6819","nonce":4292626130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6821","nonce":4292626130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6825","nonce":4292626130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4292626130},{"type":"v1","addr":"172.21.15.86:6823","nonce":4292626130}]},"public_addr":"172.21.15.86:6819/4292626130","cluster_addr":"172.21.15.86:6821/4292626130","heartbeat_back_addr":"172.21.15.86:6825/4292626130","heartbeat_front_addr":"172.21.15.86:6823/4292626130","state":["exists","up"]},{"osd":7,"uuid":"3e490f98-55e1-44e0-8c95-234d0d2b05d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6817","nonce":1354625661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6819","nonce":1354625661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6823","nonce":1354625661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1354625661},{"type":"v1","addr":"172.21.15.131:6821","nonce":1354625661}]},"public_addr":"172.21.15.131:6817/1354625661","cluster_addr":"172.21.15.131:6819/1354625661","heartbeat_back_addr":"172.21.15.131:6823/1354625661","heartbeat_front_addr":"172.21.15.131:6821/1354625661","state":["exists","up"]},{"osd":8,"uuid":"e3b7bdec-dde1-4853-9969-3de47c4ba84e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6817","nonce":2277591889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6819","nonce":2277591889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6823","nonce":2277591889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2277591889},{"type":"v1","addr":"172.21.15.167:6821","nonce":2277591889}]},"public_addr":"172.21.15.167:6817/2277591889","cluster_addr":"172.21.15.167:6819/2277591889","heartbeat_back_addr":"172.21.15.167:6823/2277591889","heartbeat_front_addr":"172.21.15.167:6821/2277591889","state":["exists","up"]},{"osd":9,"uuid":"285c5d25-0d34-4b60-a709-e8ba94d47a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6827","nonce":1269690910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6829","nonce":1269690910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6833","nonce":1269690910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1269690910},{"type":"v1","addr":"172.21.15.86:6831","nonce":1269690910}]},"public_addr":"172.21.15.86:6827/1269690910","cluster_addr":"172.21.15.86:6829/1269690910","heartbeat_back_addr":"172.21.15.86:6833/1269690910","heartbeat_front_addr":"172.21.15.86:6831/1269690910","state":["exists","up"]},{"osd":10,"uuid":"2c9eab90-57b4-4b6c-bed1-8f589dc8d971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6825","nonce":407606440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6827","nonce":407606440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6831","nonce":407606440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":407606440},{"type":"v1","addr":"172.21.15.131:6829","nonce":407606440}]},"public_addr":"172.21.15.131:6825/407606440","cluster_addr":"172.21.15.131:6827/407606440","heartbeat_back_addr":"172.21.15.131:6831/407606440","heartbeat_front_addr":"172.21.15.131:6829/407606440","state":["exists","up"]},{"osd":11,"uuid":"1101cd32-ff32-47ff-8023-01e657e68b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6825","nonce":736141441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6827","nonce":736141441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6831","nonce":736141441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":736141441},{"type":"v1","addr":"172.21.15.167:6829","nonce":736141441}]},"public_addr":"172.21.15.167:6825/736141441","cluster_addr":"172.21.15.167:6827/736141441","heartbeat_back_addr":"172.21.15.167:6831/736141441","heartbeat_front_addr":"172.21.15.167:6829/736141441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:07.846420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:25.996140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:05:43.558134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:04.548706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:22.513050+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:06:40.451492+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:01.529958+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:19.097777+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:36.907834+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:07:58.190789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:16.102686+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T19:08:33.378942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1950872310":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/3835111951":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6800/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6800/2290148199":"2024-03-23T19:03:25.120388+0000","172.21.15.86:6801/2748771516":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/2112495366":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/764935385":"2024-03-23T19:03:50.084047+0000","172.21.15.86:0/583320025":"2024-03-23T19:03:25.120388+0000","172.21.15.86:0/1207258255":"2024-03-23T19:03:50.084047+0000","172.21.15.86:6801/2290148199":"2024-03-23T19:03:25.120388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T19:20:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a140a0a90 msgr2=0x7f7a140a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a140a0a90 0x7f7a140a2f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7a300fe5c0 tx=0x7f7a20009260 comp rx=0 tx=0).stop 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 msgr2=0x7f7a30197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 0x7f7a30197220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800e9a0 tx=0x7f7a1800ecb0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 shutdown_connections 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7a140a0a90 0x7f7a140a2f50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a301027e0 0x7f7a3019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30100c50 0x7f7a30196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 --2- 172.21.15.86:0/918221296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a30101e40 0x7f7a30197220 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 >> 172.21.15.86:0/918221296 conn(0x7f7a300fc1c0 msgr2=0x7f7a301051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 shutdown_connections 2024-03-22T19:20:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.600+0000 7f7a36a00700 1 -- 172.21.15.86:0/918221296 wait complete. 2024-03-22T19:20:12.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T19:20:12.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/211390465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:20:12.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:12 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/884010684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:20:12.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/211390465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:20:12.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:12 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/884010684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 -- 172.21.15.86:0/2273255033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 msgr2=0x7f1790109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 --2- 172.21.15.86:0/2273255033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f1790109190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f178800b0d0 tx=0x7f178800b3e0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 -- 172.21.15.86:0/2273255033 shutdown_connections 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 --2- 172.21.15.86:0/2273255033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f1790109190 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 --2- 172.21.15.86:0/2273255033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1790100d50 0x7f1790101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 --2- 172.21.15.86:0/2273255033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1790101f40 0x7f17901023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 -- 172.21.15.86:0/2273255033 >> 172.21.15.86:0/2273255033 conn(0x7f17900fc2c0 msgr2=0x7f17900fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 -- 172.21.15.86:0/2273255033 shutdown_connections 2024-03-22T19:20:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.813+0000 7f1798000700 1 -- 172.21.15.86:0/2273255033 wait complete. 2024-03-22T19:20:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 Processor -- start 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 -- start start 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1790100d50 0x7f1790196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1790101f40 0x7f1790197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f179019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019e8f0 con 0x7f17901028e0 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019ea30 con 0x7f1790100d50 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.814+0000 7f1798000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019ebc0 con 0x7f1790101f40 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f179019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f179019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36518/0 (socket says 172.21.15.86:36518) 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 -- 172.21.15.86:0/2379012369 learned_addr learned my addr 172.21.15.86:0/2379012369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1795600700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1790100d50 0x7f1790196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1794c00700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1790101f40 0x7f1790197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 -- 172.21.15.86:0/2379012369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1790101f40 msgr2=0x7f1790197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1790101f40 0x7f1790197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 -- 172.21.15.86:0/2379012369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1790100d50 msgr2=0x7f1790196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1790100d50 0x7f1790196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 -- 172.21.15.86:0/2379012369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009d00 con 0x7f17901028e0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.815+0000 7f1796000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f179019e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1788007b60 tx=0x7f1788008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.816+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800e040 con 0x7f17901028e0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.816+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f178801d930 con 0x7f17901028e0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.816+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179019ee40 con 0x7f17901028e0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.816+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178801b9f0 con 0x7f17901028e0 2024-03-22T19:20:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.816+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179019f210 con 0x7f17901028e0 2024-03-22T19:20:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.817+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790108100 con 0x7f17901028e0 2024-03-22T19:20:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.819+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f178801daa0 con 0x7f17901028e0 2024-03-22T19:20:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.820+0000 7f178e000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17780a0a90 0x7f17780a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.820+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8161+0+0 (secure 0 0 0) 0x7f17880cdbc0 con 0x7f17901028e0 2024-03-22T19:20:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.820+0000 7f1795600700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17780a0a90 0x7f17780a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.821+0000 7f1795600700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17780a0a90 0x7f17780a2f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1790101da0 tx=0x7f1784009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:12.822+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1788097610 con 0x7f17901028e0 2024-03-22T19:20:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/211390465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:20:12.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:12 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/884010684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T19:20:13.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.076+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f179002d130 con 0x7f17901028e0 2024-03-22T19:20:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.102+0000 7f178e000700 1 -- 172.21.15.86:0/2379012369 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f179002d130 con 0x7f17901028e0 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17780a0a90 msgr2=0x7f17780a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17780a0a90 0x7f17780a2f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1790101da0 tx=0x7f1784009260 comp rx=0 tx=0).stop 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 msgr2=0x7f179019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f179019e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1788007b60 tx=0x7f1788008e40 comp rx=0 tx=0).stop 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 shutdown_connections 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f17780a0a90 0x7f17780a2f50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17901028e0 0x7f179019e320 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1790100d50 0x7f1790196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 --2- 172.21.15.86:0/2379012369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1790101f40 0x7f1790197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 >> 172.21.15.86:0/2379012369 conn(0x7f17900fc2c0 msgr2=0x7f1790105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 shutdown_connections 2024-03-22T19:20:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.105+0000 7f1798000700 1 -- 172.21.15.86:0/2379012369 wait complete. 2024-03-22T19:20:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T19:20:13.126 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T19:20:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 -- 172.21.15.86:0/1104443186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58102720 msgr2=0x7faa58108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 --2- 172.21.15.86:0/1104443186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58102720 0x7faa58108fd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b0d0 tx=0x7faa5000b3e0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 -- 172.21.15.86:0/1104443186 shutdown_connections 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 --2- 172.21.15.86:0/1104443186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58102720 0x7faa58108fd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 --2- 172.21.15.86:0/1104443186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58100b90 0x7faa58100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 --2- 172.21.15.86:0/1104443186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58101d80 0x7faa581021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 -- 172.21.15.86:0/1104443186 >> 172.21.15.86:0/1104443186 conn(0x7faa580fc140 msgr2=0x7faa580fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 -- 172.21.15.86:0/1104443186 shutdown_connections 2024-03-22T19:20:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.334+0000 7faa5d600700 1 -- 172.21.15.86:0/1104443186 wait complete. 2024-03-22T19:20:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.335+0000 7faa5d600700 1 Processor -- start 2024-03-22T19:20:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.335+0000 7faa5d600700 1 -- start start 2024-03-22T19:20:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58100b90 0x7faa58198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 0x7faa581993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5d600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58102720 0x7faa581a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581a0950 con 0x7faa58100b90 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5d600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581a0a90 con 0x7faa58101d80 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5d600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581a0c20 con 0x7faa58102720 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58100b90 0x7faa58198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 0x7faa581993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 0x7faa581993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44240/0 (socket says 172.21.15.86:44240) 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 -- 172.21.15.86:0/2345010205 learned_addr learned my addr 172.21.15.86:0/2345010205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5cc00700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58102720 0x7faa581a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58100b90 0x7faa58198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36556/0 (socket says 172.21.15.86:36556) 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 -- 172.21.15.86:0/2345010205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58102720 msgr2=0x7faa581a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58102720 0x7faa581a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 -- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58100b90 msgr2=0x7faa58198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58100b90 0x7faa58198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa57400700 1 -- 172.21.15.86:0/2345010205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa50009d00 con 0x7faa58101d80 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.336+0000 7faa5cc00700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58102720 0x7faa581a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.337+0000 7faa57400700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 0x7faa581993b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa4400e9a0 tx=0x7faa4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.337+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44009790 con 0x7faa58101d80 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.337+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa581a0f00 con 0x7faa58101d80 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.337+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa44010430 con 0x7faa58101d80 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.337+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44018680 con 0x7faa58101d80 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.337+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa581a1380 con 0x7faa58101d80 2024-03-22T19:20:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.338+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa58067dc0 con 0x7faa58101d80 2024-03-22T19:20:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.340+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7faa440105a0 con 0x7faa58101d80 2024-03-22T19:20:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.341+0000 7faa54c00700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faa3c0a0a90 0x7faa3c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.341+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8591+0+0 (secure 0 0 0) 0x7faa44014070 con 0x7faa58101d80 2024-03-22T19:20:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.341+0000 7faa57e00700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faa3c0a0a90 0x7faa3c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.342+0000 7faa57e00700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faa3c0a0a90 0x7faa3c0a2f50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faa58101be0 tx=0x7faa48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:13.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.344+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa44095da0 con 0x7faa58101d80 2024-03-22T19:20:13.456 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a[38009]: 2024-03-22T19:20:13.094+0000 7f60e6800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a[90448]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a[90448]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mds[90471]: -- 172.21.15.86:0/443881670 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f6a65c81a0 con 0x55f6a65a0c00 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a[90448]: starting mds.a at 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g[90061]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g[90061]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mds[90084]: -- 172.21.15.86:0/3780028234 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dfc059e1a0 con 0x55dfc0576800 2024-03-22T19:20:13.457 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g[90061]: starting mds.g at 2024-03-22T19:20:13.458 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h[116261]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.458 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h[116261]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.458 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h[116261]: starting mds.h at 2024-03-22T19:20:13.458 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k[115576]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.458 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k[115576]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.458 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k[115576]: starting mds.k at 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mds[116628]: -- 172.21.15.131:0/1471394595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d9ed3301a0 con 0x55d9ed306800 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: starting mds.b at 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e[115919]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e[115919]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.459 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e[115919]: starting mds.e at 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c[85019]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c[85019]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mds[85042]: -- 172.21.15.167:0/745036233 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5585216781a0 con 0x55852164e800 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c[85019]: starting mds.c at 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f[84676]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f[84676]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mds[84699]: -- 172.21.15.167:0/684605724 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ca8bd1a1a0 con 0x55ca8bcf0c00 2024-03-22T19:20:13.460 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f[84676]: starting mds.f at 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i[85359]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i[85359]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mds[85382]: -- 172.21.15.167:0/3508037760 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562d1ade01a0 con 0x562d1adb7000 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i[85359]: starting mds.i at 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l[84336]: ignoring --setuser ceph since I am not root 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l[84336]: ignoring --setgroup ceph since I am not root 2024-03-22T19:20:13.461 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l[84336]: starting mds.l at 2024-03-22T19:20:13.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:13.597+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faa581a1660 con 0x7faa58101d80 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: pgmap v563: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/918221296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2379012369' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: osdmap e83: 12 total, 12 up, 12 in 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2379012369' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T19:20:13.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:13 smithi131 ceph-mon[78559]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: pgmap v563: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/918221296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2379012369' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: osdmap e83: 12 total, 12 up, 12 in 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2379012369' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T19:20:13.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:13 smithi086 ceph-mon[38032]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-22T19:20:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: pgmap v563: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-22T19:20:13.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/918221296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T19:20:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2379012369' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T19:20:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T19:20:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T19:20:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: osdmap e83: 12 total, 12 up, 12 in 2024-03-22T19:20:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2379012369' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T19:20:13.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:13 smithi167 ceph-mon[47728]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-22T19:20:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.105+0000 7faa54c00700 1 -- 172.21.15.86:0/2345010205 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa440993c0 con 0x7faa58101d80 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faa3c0a0a90 msgr2=0x7faa3c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faa3c0a0a90 0x7faa3c0a2f50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faa58101be0 tx=0x7faa48009260 comp rx=0 tx=0).stop 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 msgr2=0x7faa581993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 0x7faa581993b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa4400e9a0 tx=0x7faa4400ecb0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 shutdown_connections 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7faa3c0a0a90 0x7faa3c0a2f50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa58102720 0x7faa581a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa58100b90 0x7faa58198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 --2- 172.21.15.86:0/2345010205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa58101d80 0x7faa581993b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 >> 172.21.15.86:0/2345010205 conn(0x7faa580fc140 msgr2=0x7faa58107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 shutdown_connections 2024-03-22T19:20:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.108+0000 7faa5d600700 1 -- 172.21.15.86:0/2345010205 wait complete. 2024-03-22T19:20:14.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T19:20:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 -- 172.21.15.86:0/1310122364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb01040e0 msgr2=0x7f7eb01064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/1310122364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb01040e0 0x7f7eb01064d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009990 tx=0x7f7e9c009ca0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 -- 172.21.15.86:0/1310122364 shutdown_connections 2024-03-22T19:20:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/1310122364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb0106a10 0x7f7eb0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/1310122364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7eb01017b0 0x7f7eb0103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/1310122364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb01040e0 0x7f7eb01064d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 -- 172.21.15.86:0/1310122364 >> 172.21.15.86:0/1310122364 conn(0x7f7eb00fb1a0 msgr2=0x7f7eb00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 -- 172.21.15.86:0/1310122364 shutdown_connections 2024-03-22T19:20:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.332+0000 7f7eb4a00700 1 -- 172.21.15.86:0/1310122364 wait complete. 2024-03-22T19:20:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 Processor -- start 2024-03-22T19:20:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 -- start start 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7eb01017b0 0x7f7eb0196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 0x7f7eb0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb0106a10 0x7f7eb019e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019e7b0 con 0x7f7eb0106a10 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019e920 con 0x7f7eb01040e0 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.334+0000 7f7eb4a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019eae0 con 0x7f7eb01017b0 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 0x7f7eb0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7eae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb0106a10 0x7f7eb019e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 0x7f7eb0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.86:44254/0 (socket says 172.21.15.86:44254) 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 -- 172.21.15.86:0/830354816 learned_addr learned my addr 172.21.15.86:0/830354816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7eade00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7eb01017b0 0x7f7eb0196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 -- 172.21.15.86:0/830354816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7eb01017b0 msgr2=0x7f7eb0196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7eb01017b0 0x7f7eb0196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 -- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb0106a10 msgr2=0x7f7eb019e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb0106a10 0x7f7eb019e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 -- 172.21.15.86:0/830354816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0095f0 con 0x7f7eb01040e0 2024-03-22T19:20:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7eae800700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb0106a10 0x7f7eb019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ead400700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 0x7f7eb0197200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c000c00 tx=0x7f7e9c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.335+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c01d070 con 0x7f7eb01040e0 2024-03-22T19:20:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.336+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb019ed60 con 0x7f7eb01040e0 2024-03-22T19:20:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.336+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e9c021410 con 0x7f7eb01040e0 2024-03-22T19:20:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.336+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c017650 con 0x7f7eb01040e0 2024-03-22T19:20:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.336+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb019f180 con 0x7f7eb01040e0 2024-03-22T19:20:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.337+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb00fcd20 con 0x7f7eb01040e0 2024-03-22T19:20:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.339+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7f7e9c0217f0 con 0x7f7eb01040e0 2024-03-22T19:20:14.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.340+0000 7f7ea6a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e940a0a90 0x7f7e940a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:14.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.340+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8591+0+0 (secure 0 0 0) 0x7f7e9c0cdcf0 con 0x7f7eb01040e0 2024-03-22T19:20:14.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.340+0000 7f7eade00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e940a0a90 0x7f7e940a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:14.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.341+0000 7f7eade00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e940a0a90 0x7f7e940a2f50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0198140 tx=0x7f7ea00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.343+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7e9c097590 con 0x7f7eb01040e0 2024-03-22T19:20:14.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:14.551+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7eb002d0b0 con 0x7f7eb01040e0 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2345010205' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: Cluster is now healthy 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6833/3454224260,v1:172.21.15.167:6835/3454224260] up:boot 2024-03-22T19:20:14.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6836/653832190,v1:172.21.15.167:6837/653832190] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6838/913492020,v1:172.21.15.167:6839/913492020] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6836/2795405625,v1:172.21.15.131:6838/2795405625] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6837/1434403327,v1:172.21.15.131:6839/1434403327] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.167:6832/3944315019,v1:172.21.15.167:6834/3944315019] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6832/2103307928,v1:172.21.15.131:6833/2103307928] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.131:6834/2690751518,v1:172.21.15.131:6835/2690751518] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6838/3647167914,v1:172.21.15.86:6840/3647167914] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: mds.? [v2:172.21.15.86:6839/2891015319,v1:172.21.15.86:6841/2891015319] up:boot 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: fsmap 12 up:standby 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-22T19:20:14.723 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-22T19:20:14.724 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:14 smithi131 ceph-mon[78559]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-22T19:20:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2345010205' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:20:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:20:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T19:20:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T19:20:14.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: Cluster is now healthy 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6833/3454224260,v1:172.21.15.167:6835/3454224260] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6836/653832190,v1:172.21.15.167:6837/653832190] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6838/913492020,v1:172.21.15.167:6839/913492020] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6836/2795405625,v1:172.21.15.131:6838/2795405625] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6837/1434403327,v1:172.21.15.131:6839/1434403327] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.167:6832/3944315019,v1:172.21.15.167:6834/3944315019] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6832/2103307928,v1:172.21.15.131:6833/2103307928] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.131:6834/2690751518,v1:172.21.15.131:6835/2690751518] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6838/3647167914,v1:172.21.15.86:6840/3647167914] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: mds.? [v2:172.21.15.86:6839/2891015319,v1:172.21.15.86:6841/2891015319] up:boot 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: fsmap 12 up:standby 2024-03-22T19:20:14.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-22T19:20:14.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:14 smithi086 ceph-mon[38032]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-22T19:20:14.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2345010205' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: Cluster is now healthy 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6833/3454224260,v1:172.21.15.167:6835/3454224260] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6836/653832190,v1:172.21.15.167:6837/653832190] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6838/913492020,v1:172.21.15.167:6839/913492020] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6836/2795405625,v1:172.21.15.131:6838/2795405625] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6837/1434403327,v1:172.21.15.131:6839/1434403327] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.167:6832/3944315019,v1:172.21.15.167:6834/3944315019] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6832/2103307928,v1:172.21.15.131:6833/2103307928] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.131:6834/2690751518,v1:172.21.15.131:6835/2690751518] up:boot 2024-03-22T19:20:14.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6838/3647167914,v1:172.21.15.86:6840/3647167914] up:boot 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: mds.? [v2:172.21.15.86:6839/2891015319,v1:172.21.15.86:6841/2891015319] up:boot 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: fsmap 12 up:standby 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-22T19:20:14.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:14 smithi167 ceph-mon[47728]: from='mgr.14152 172.21.15.86:0/3609912750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-22T19:20:15.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.473+0000 7f7ea6a00700 1 -- 172.21.15.86:0/830354816 <== mon.2 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7e9c095e60 con 0x7f7eb01040e0 2024-03-22T19:20:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e940a0a90 msgr2=0x7f7e940a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e940a0a90 0x7f7e940a2f50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0198140 tx=0x7f7ea00073d0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 msgr2=0x7f7eb0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 0x7f7eb0197200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c000c00 tx=0x7f7e9c005740 comp rx=0 tx=0).stop 2024-03-22T19:20:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 shutdown_connections 2024-03-22T19:20:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7f7e940a0a90 0x7f7e940a2f50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb0106a10 0x7f7eb019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7eb01017b0 0x7f7eb0196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 --2- 172.21.15.86:0/830354816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb01040e0 0x7f7eb0197200 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 >> 172.21.15.86:0/830354816 conn(0x7f7eb00fb1a0 msgr2=0x7f7eb0103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.477+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 shutdown_connections 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.478+0000 7f7eb4a00700 1 -- 172.21.15.86:0/830354816 wait complete. 2024-03-22T19:20:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T19:20:15.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T19:20:15.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 -- 172.21.15.86:0/7233612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c108c10 msgr2=0x7ffa0c10b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 --2- 172.21.15.86:0/7233612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c108c10 0x7ffa0c10b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b0a0 tx=0x7ffa0400b3b0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 -- 172.21.15.86:0/7233612 shutdown_connections 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 --2- 172.21.15.86:0/7233612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c108c10 0x7ffa0c10b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 --2- 172.21.15.86:0/7233612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa0c074df0 0x7ffa0c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 --2- 172.21.15.86:0/7233612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 -- 172.21.15.86:0/7233612 >> 172.21.15.86:0/7233612 conn(0x7ffa0c0fc2c0 msgr2=0x7ffa0c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 -- 172.21.15.86:0/7233612 shutdown_connections 2024-03-22T19:20:15.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.699+0000 7ffa12e00700 1 -- 172.21.15.86:0/7233612 wait complete. 2024-03-22T19:20:15.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.700+0000 7ffa12e00700 1 Processor -- start 2024-03-22T19:20:15.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 -- start start 2024-03-22T19:20:15.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:15.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c074df0 0x7ffa0c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa0c108c10 0x7ffa0c1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a0b20 con 0x7ffa0c073810 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a0c60 con 0x7ffa0c108c10 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa12e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a0df0 con 0x7ffa0c074df0 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c074df0 0x7ffa0c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36580/0 (socket says 172.21.15.86:36580) 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 -- 172.21.15.86:0/2083678767 learned_addr learned my addr 172.21.15.86:0/2083678767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa10e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa0c108c10 0x7ffa0c1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 -- 172.21.15.86:0/2083678767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c074df0 msgr2=0x7ffa0c199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c074df0 0x7ffa0c199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 -- 172.21.15.86:0/2083678767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa0c108c10 msgr2=0x7ffa0c1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.701+0000 7ffa0be00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa0c108c10 0x7ffa0c1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa0be00700 1 -- 172.21.15.86:0/2083678767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8009520 con 0x7ffa0c073810 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa0b400700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c074df0 0x7ffa0c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa0be00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c198ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00eb30 tx=0x7ff9fc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc009880 con 0x7ffa0c073810 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc009520 con 0x7ffa0c073810 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc0044e0 con 0x7ffa0c073810 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa04009d00 con 0x7ffa0c073810 2024-03-22T19:20:15.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.702+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc003d60 con 0x7ffa0c073810 2024-03-22T19:20:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.703+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c1079d0 con 0x7ffa0c073810 2024-03-22T19:20:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.705+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135388+0+0 (secure 0 0 0) 0x7ff9fc003f00 con 0x7ffa0c073810 2024-03-22T19:20:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.706+0000 7ffa08c00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9f00a0a90 0x7ff9f00a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T19:20:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.706+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8058+0+0 (secure 0 0 0) 0x7ff9fc014070 con 0x7ffa0c073810 2024-03-22T19:20:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.706+0000 7ffa0b400700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9f00a0a90 0x7ff9f00a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T19:20:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.707+0000 7ffa0b400700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9f00a0a90 0x7ff9f00a2f50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T19:20:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.710+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff9fc095ec0 con 0x7ffa0c073810 2024-03-22T19:20:15.721 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:15 smithi131 ceph-mon[78559]: pgmap v565: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 818 B/s wr, 11 op/s 2024-03-22T19:20:15.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:15 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/830354816' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:15.722 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:15 smithi131 ceph-mon[78559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:15 smithi086 ceph-mon[38032]: pgmap v565: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 818 B/s wr, 11 op/s 2024-03-22T19:20:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:15 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/830354816' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:15 smithi086 ceph-mon[38032]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:15 smithi167 ceph-mon[47728]: pgmap v565: 97 pgs: 97 active+clean; 25 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 818 B/s wr, 11 op/s 2024-03-22T19:20:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:15 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/830354816' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:15 smithi167 ceph-mon[47728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:15.943+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa0c067ac0 con 0x7ffa0c073810 2024-03-22T19:20:16.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.476+0000 7ffa08c00700 1 -- 172.21.15.86:0/2083678767 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff9fc0b9030 con 0x7ffa0c073810 2024-03-22T19:20:16.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.485+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9f00a0a90 msgr2=0x7ff9f00a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:16.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.485+0000 7ffa12e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9f00a0a90 0x7ff9f00a2f50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f8009260 comp rx=0 tx=0).stop 2024-03-22T19:20:16.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.485+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 msgr2=0x7ffa0c198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.485+0000 7ffa12e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c198ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00eb30 tx=0x7ff9fc00ee40 comp rx=0 tx=0).stop 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.486+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 shutdown_connections 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.486+0000 7ffa12e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:6800/1126645338,v1:172.21.15.86:6801/1126645338] conn(0x7ff9f00a0a90 0x7ff9f00a2f50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.486+0000 7ffa12e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa0c108c10 0x7ffa0c1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.486+0000 7ffa12e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0c073810 0x7ffa0c198ff0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.486+0000 7ffa12e00700 1 --2- 172.21.15.86:0/2083678767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa0c074df0 0x7ffa0c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.486+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 >> 172.21.15.86:0/2083678767 conn(0x7ffa0c0fc2c0 msgr2=0x7ffa0c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T19:20:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.487+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 shutdown_connections 2024-03-22T19:20:16.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T19:20:16.489+0000 7ffa12e00700 1 -- 172.21.15.86:0/2083678767 wait complete. 2024-03-22T19:20:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T19:20:16.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:20:16.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:20:16.739 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:16 smithi131 ceph-mon[78559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T19:20:16.739 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:16 smithi131 ceph-mon[78559]: osdmap e84: 12 total, 12 up, 12 in 2024-03-22T19:20:16.739 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:16 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2083678767' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:16.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:16 smithi086 ceph-mon[38032]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T19:20:16.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:16 smithi086 ceph-mon[38032]: osdmap e84: 12 total, 12 up, 12 in 2024-03-22T19:20:16.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:16 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2083678767' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:16.769 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:20:16.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:16 smithi167 ceph-mon[47728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T19:20:16.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:16 smithi167 ceph-mon[47728]: osdmap e84: 12 total, 12 up, 12 in 2024-03-22T19:20:16.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:16 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2083678767' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T19:20:16.849 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-22T19:20:16.849 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T19:20:16.892 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T19:20:16.965 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T19:20:16.982 INFO:tasks.cephadm:Stopping all daemons... 2024-03-22T19:20:16.982 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-22T19:20:16.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a 2024-03-22T19:20:17.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:17 smithi086 systemd[1]: Stopping Ceph mon.a for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:17.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:17 smithi086 ceph-mon[38032]: pgmap v567: 65 pgs: 65 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:20:17.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:17 smithi086 ceph-mon[38032]: from='client.? 172.21.15.86:0/2083678767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T19:20:17.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:17 smithi086 ceph-mon[38032]: osdmap e85: 12 total, 12 up, 12 in 2024-03-22T19:20:17.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:17 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a[38009]: 2024-03-22T19:20:17.700+0000 7f60ed600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:17.760 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:17 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a[38009]: 2024-03-22T19:20:17.700+0000 7f60ed600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-22T19:20:17.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:17 smithi167 ceph-mon[47728]: pgmap v567: 65 pgs: 65 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:20:17.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:17 smithi167 ceph-mon[47728]: from='client.? 172.21.15.86:0/2083678767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T19:20:17.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:17 smithi167 ceph-mon[47728]: osdmap e85: 12 total, 12 up, 12 in 2024-03-22T19:20:17.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:17 smithi131 ceph-mon[78559]: pgmap v567: 65 pgs: 65 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-22T19:20:17.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:17 smithi131 ceph-mon[78559]: from='client.? 172.21.15.86:0/2083678767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T19:20:17.972 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:17 smithi131 ceph-mon[78559]: osdmap e85: 12 total, 12 up, 12 in 2024-03-22T19:20:18.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 22 19:20:18 smithi086 bash[116538]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-a 2024-03-22T19:20:18.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.a.service' 2024-03-22T19:20:18.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:18.714 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-22T19:20:18.714 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-22T19:20:18.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.b 2024-03-22T19:20:19.015 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:18 smithi131 systemd[1]: Stopping Ceph mon.b for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:19.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:19 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-b[78536]: 2024-03-22T19:20:19.012+0000 7f20b3b04700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:19.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:19 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-b[78536]: 2024-03-22T19:20:19.012+0000 7f20b3b04700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-22T19:20:19.646 INFO:journalctl@ceph.mon.b.smithi131.stdout:Mar 22 19:20:19 smithi131 bash[120921]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-b 2024-03-22T19:20:19.653 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.b.service' 2024-03-22T19:20:19.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:19.716 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-22T19:20:19.716 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-22T19:20:19.716 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.c 2024-03-22T19:20:20.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:19 smithi167 systemd[1]: Stopping Ceph mon.c for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:20.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:19 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-c[47705]: 2024-03-22T19:20:19.989+0000 7f3b7509e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:20.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:19 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-c[47705]: 2024-03-22T19:20:19.989+0000 7f3b7509e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-22T19:20:20.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Mar 22 19:20:20 smithi167 bash[89756]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mon-c 2024-03-22T19:20:20.704 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mon.c.service' 2024-03-22T19:20:20.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:20.760 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-22T19:20:20.760 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-22T19:20:20.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x 2024-03-22T19:20:21.159 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:20:20 smithi086 systemd[1]: Stopping Ceph mgr.x for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:21.431 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:20:21 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x[38282]: 2024-03-22T19:20:21.156+0000 7fde2e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:21.759 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 22 19:20:21 smithi086 bash[116675]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-x 2024-03-22T19:20:21.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-22T19:20:21.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:21.945 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-22T19:20:21.945 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-22T19:20:21.945 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.y 2024-03-22T19:20:22.369 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:20:21 smithi131 systemd[1]: Stopping Ceph mgr.y for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:22.722 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:20:22 smithi131 bash[121071]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-y 2024-03-22T19:20:22.722 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Mar 22 19:20:22 smithi131 systemd[1]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-22T19:20:22.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-22T19:20:22.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:22.771 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-22T19:20:22.771 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-22T19:20:22.771 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.z 2024-03-22T19:20:23.078 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:20:22 smithi167 systemd[1]: Stopping Ceph mgr.z for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:23.439 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:20:23 smithi167 bash[89909]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mgr-z 2024-03-22T19:20:23.439 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Mar 22 19:20:23 smithi167 systemd[1]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-22T19:20:23.445 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-22T19:20:23.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:23.481 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-22T19:20:23.481 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-22T19:20:23.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.0 2024-03-22T19:20:23.849 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:23 smithi086 systemd[1]: Stopping Ceph osd.0 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:24.259 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:23 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0[56811]: 2024-03-22T19:20:23.846+0000 7f76ac000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:24.259 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:23 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0[56811]: 2024-03-22T19:20:23.846+0000 7f76ac000700 -1 osd.0 85 *** Got signal Terminated *** 2024-03-22T19:20:24.259 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:23 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0[56811]: 2024-03-22T19:20:23.846+0000 7f76ac000700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:20:30.259 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:29 smithi086 bash[116815]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-0 2024-03-22T19:20:31.713 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.0.service' 2024-03-22T19:20:31.773 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:31 smithi086 systemd[1]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.0.service: Succeeded. 2024-03-22T19:20:31.773 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 22 19:20:31 smithi086 systemd[1]: Stopped Ceph osd.0 for b22c63ac-e87e-11ee-95cd-87774f69a715. 2024-03-22T19:20:31.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:31.802 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-22T19:20:31.802 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-22T19:20:31.802 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.1 2024-03-22T19:20:32.222 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:20:31 smithi131 systemd[1]: Stopping Ceph osd.1 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:32.222 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:20:32 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1[83530]: 2024-03-22T19:20:32.044+0000 7f3794136700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:32.222 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:20:32 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1[83530]: 2024-03-22T19:20:32.044+0000 7f3794136700 -1 osd.1 85 *** Got signal Terminated *** 2024-03-22T19:20:32.222 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:20:32 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1[83530]: 2024-03-22T19:20:32.044+0000 7f3794136700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:20:37.972 INFO:journalctl@ceph.osd.1.smithi131.stdout:Mar 22 19:20:37 smithi131 bash[121215]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-1 2024-03-22T19:20:39.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.1.service' 2024-03-22T19:20:39.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:39.256 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-22T19:20:39.256 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-22T19:20:39.256 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.2 2024-03-22T19:20:39.579 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:20:39 smithi167 systemd[1]: Stopping Ceph osd.2 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:39.579 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:20:39 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2[53482]: 2024-03-22T19:20:39.487+0000 7f79f233b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:39.579 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:20:39 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2[53482]: 2024-03-22T19:20:39.487+0000 7f79f233b700 -1 osd.2 85 *** Got signal Terminated *** 2024-03-22T19:20:39.579 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:20:39 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2[53482]: 2024-03-22T19:20:39.487+0000 7f79f233b700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:20:45.578 INFO:journalctl@ceph.osd.2.smithi167.stdout:Mar 22 19:20:45 smithi167 bash[90062]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-2 2024-03-22T19:20:46.476 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.2.service' 2024-03-22T19:20:46.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:46.517 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-22T19:20:46.517 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-22T19:20:46.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.3 2024-03-22T19:20:46.889 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:20:46 smithi086 systemd[1]: Stopping Ceph osd.3 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:47.259 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:20:46 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3[65372]: 2024-03-22T19:20:46.886+0000 7fcb8b000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:47.259 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:20:46 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3[65372]: 2024-03-22T19:20:46.886+0000 7fcb8b000700 -1 osd.3 85 *** Got signal Terminated *** 2024-03-22T19:20:47.259 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:20:46 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3[65372]: 2024-03-22T19:20:46.886+0000 7fcb8b000700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:20:53.259 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 22 19:20:52 smithi086 bash[117089]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-3 2024-03-22T19:20:54.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.3.service' 2024-03-22T19:20:54.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:20:54.641 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-22T19:20:54.641 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-22T19:20:54.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.4 2024-03-22T19:20:54.972 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:20:54 smithi131 systemd[1]: Stopping Ceph osd.4 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:20:54.972 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:20:54 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4[92175]: 2024-03-22T19:20:54.873+0000 7efed23ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:20:54.972 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:20:54 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4[92175]: 2024-03-22T19:20:54.873+0000 7efed23ed700 -1 osd.4 85 *** Got signal Terminated *** 2024-03-22T19:20:54.972 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:20:54 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4[92175]: 2024-03-22T19:20:54.873+0000 7efed23ed700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:00.722 INFO:journalctl@ceph.osd.4.smithi131.stdout:Mar 22 19:21:00 smithi131 bash[121477]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-4 2024-03-22T19:21:01.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.4.service' 2024-03-22T19:21:01.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:01.747 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-22T19:21:01.747 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-22T19:21:01.747 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.5 2024-03-22T19:21:02.078 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:21:01 smithi167 systemd[1]: Stopping Ceph osd.5 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:02.078 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:21:01 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5[62105]: 2024-03-22T19:21:01.991+0000 7ffa7e925700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:02.079 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:21:01 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5[62105]: 2024-03-22T19:21:01.991+0000 7ffa7e925700 -1 osd.5 85 *** Got signal Terminated *** 2024-03-22T19:21:02.079 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:21:01 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5[62105]: 2024-03-22T19:21:01.991+0000 7ffa7e925700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:08.078 INFO:journalctl@ceph.osd.5.smithi167.stdout:Mar 22 19:21:07 smithi167 bash[90316]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-5 2024-03-22T19:21:09.014 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.5.service' 2024-03-22T19:21:09.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:09.055 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-22T19:21:09.055 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-22T19:21:09.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.6 2024-03-22T19:21:09.460 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:21:09 smithi086 systemd[1]: Stopping Ceph osd.6 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:09.759 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:21:09 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6[74030]: 2024-03-22T19:21:09.458+0000 7f5463a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:09.759 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:21:09 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6[74030]: 2024-03-22T19:21:09.458+0000 7f5463a00700 -1 osd.6 85 *** Got signal Terminated *** 2024-03-22T19:21:09.759 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:21:09 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6[74030]: 2024-03-22T19:21:09.458+0000 7f5463a00700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:15.509 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 22 19:21:15 smithi086 bash[117542]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-6 2024-03-22T19:21:16.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.6.service' 2024-03-22T19:21:17.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:17.060 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-22T19:21:17.060 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-22T19:21:17.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.7 2024-03-22T19:21:17.472 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:21:17 smithi131 systemd[1]: Stopping Ceph osd.7 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:17.472 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:21:17 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7[100899]: 2024-03-22T19:21:17.286+0000 7f6d52b02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:17.472 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:21:17 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7[100899]: 2024-03-22T19:21:17.286+0000 7f6d52b02700 -1 osd.7 85 *** Got signal Terminated *** 2024-03-22T19:21:17.472 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:21:17 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7[100899]: 2024-03-22T19:21:17.286+0000 7f6d52b02700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:23.222 INFO:journalctl@ceph.osd.7.smithi131.stdout:Mar 22 19:21:22 smithi131 bash[121728]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-7 2024-03-22T19:21:24.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.7.service' 2024-03-22T19:21:24.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:24.107 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-22T19:21:24.108 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-22T19:21:24.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.8 2024-03-22T19:21:24.579 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:21:24 smithi167 systemd[1]: Stopping Ceph osd.8 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:24.579 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:21:24 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8[70823]: 2024-03-22T19:21:24.347+0000 7fc1a144b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:24.579 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:21:24 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8[70823]: 2024-03-22T19:21:24.347+0000 7fc1a144b700 -1 osd.8 85 *** Got signal Terminated *** 2024-03-22T19:21:24.579 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:21:24 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8[70823]: 2024-03-22T19:21:24.347+0000 7fc1a144b700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:30.171 INFO:journalctl@ceph.osd.8.smithi167.stdout:Mar 22 19:21:29 smithi167 bash[90567]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-8 2024-03-22T19:21:30.446 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:30 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:30.168+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:31.152 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.8.service' 2024-03-22T19:21:31.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:31.193 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-22T19:21:31.193 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-22T19:21:31.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.9 2024-03-22T19:21:31.509 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:31 smithi086 systemd[1]: Stopping Ceph osd.9 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:31.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:31 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:31.211+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:31.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:31 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:31.680+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:32.009 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:31 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:21:31.580+0000 7fa2a1600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:32.009 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:31 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:21:31.580+0000 7fa2a1600700 -1 osd.9 85 *** Got signal Terminated *** 2024-03-22T19:21:32.009 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:31 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:21:31.580+0000 7fa2a1600700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:32.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:32 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:32.233+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:32.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:32 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:32.656+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:33.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:33 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:33.267+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:33.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:33 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:33.645+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:34.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:34 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:34.238+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:34.759 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:34 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:21:34.291+0000 7fa297e00700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.143199+0000 front 2024-03-22T19:20:28.143246+0000 (oldest deadline 2024-03-22T19:21:34.043438+0000) 2024-03-22T19:21:34.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:34 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:34.632+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:35.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:35 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:35.212+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:35.759 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:35 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:21:35.273+0000 7fa297e00700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.143199+0000 front 2024-03-22T19:20:28.143246+0000 (oldest deadline 2024-03-22T19:21:34.043438+0000) 2024-03-22T19:21:35.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:35 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:35.594+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:36.509 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:36 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9[82716]: 2024-03-22T19:21:36.256+0000 7fa297e00700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.143199+0000 front 2024-03-22T19:20:28.143246+0000 (oldest deadline 2024-03-22T19:21:34.043438+0000) 2024-03-22T19:21:36.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:36 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:36.222+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:36.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:36 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:36.607+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:37.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:37 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:37.178+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:37.759 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 22 19:21:37 smithi086 bash[117803]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-9 2024-03-22T19:21:37.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:37 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:37.655+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:38.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:38 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:38.182+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:38.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:38 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:38.678+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:39.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.9.service' 2024-03-22T19:21:39.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:39.114 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-22T19:21:39.114 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-22T19:21:39.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.10 2024-03-22T19:21:39.472 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:39 smithi131 systemd[1]: Stopping Ceph osd.10 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:39.472 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:39 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:39.358+0000 7f0ec560b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:39.472 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:39 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:39.358+0000 7f0ec560b700 -1 osd.10 85 *** Got signal Terminated *** 2024-03-22T19:21:39.472 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:39 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:39.358+0000 7f0ec560b700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:39.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:39 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:39.218+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:39.971 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:39 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:39.643+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:39.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:39 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:39.643+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.525700+0000 front 2024-03-22T19:20:34.525642+0000 (oldest deadline 2024-03-22T19:21:39.225776+0000) 2024-03-22T19:21:40.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:40 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:40.238+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:40.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:40 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:40.238+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:40.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:40 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:40.616+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:40.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:40 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:40.616+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.525700+0000 front 2024-03-22T19:20:34.525642+0000 (oldest deadline 2024-03-22T19:21:39.225776+0000) 2024-03-22T19:21:41.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:41 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:41.275+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:41.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:41 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:41.275+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:41.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:41 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:41.579+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:41.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:41 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:41.579+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.525700+0000 front 2024-03-22T19:20:34.525642+0000 (oldest deadline 2024-03-22T19:21:39.225776+0000) 2024-03-22T19:21:42.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:42 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:42.226+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:42.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:42 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:42.226+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:42.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:42 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:42.584+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:42.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:42 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:42.584+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.525700+0000 front 2024-03-22T19:20:34.525642+0000 (oldest deadline 2024-03-22T19:21:39.225776+0000) 2024-03-22T19:21:43.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:43 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:43.210+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:43.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:43 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:43.210+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:43.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:43 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:43.607+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:28.724427+0000 front 2024-03-22T19:20:28.724387+0000 (oldest deadline 2024-03-22T19:21:31.624488+0000) 2024-03-22T19:21:43.972 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:43 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10[109555]: 2024-03-22T19:21:43.607+0000 7f0ec03fe700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.525700+0000 front 2024-03-22T19:20:34.525642+0000 (oldest deadline 2024-03-22T19:21:39.225776+0000) 2024-03-22T19:21:44.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:44 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:44.177+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:44.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:44 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:44.177+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:45.472 INFO:journalctl@ceph.osd.10.smithi131.stdout:Mar 22 19:21:45 smithi131 bash[121985]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-10 2024-03-22T19:21:45.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:45.203+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:45.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:45 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:45.203+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:46.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.10.service' 2024-03-22T19:21:46.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:46.313 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-22T19:21:46.314 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-22T19:21:46.314 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.11 2024-03-22T19:21:46.554 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:46.233+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:46.554 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:46.233+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:46.555 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:46 smithi167 systemd[1]: Stopping Ceph osd.11 for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:46.828 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:46.552+0000 7f8263b40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:46.828 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:46.552+0000 7f8263b40700 -1 osd.11 85 *** Got signal Terminated *** 2024-03-22T19:21:46.829 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:46 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:46.552+0000 7f8263b40700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T19:21:47.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:47 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:47.235+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:47.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:47 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:47.235+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:48.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:48 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:48.241+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:48.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:48 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:48.241+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:49.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:49 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:49.212+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:49.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:49 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:49.212+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:50.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:50 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:50.217+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:50.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:50 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:50.217+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:50.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:50 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:50.217+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-03-22T19:20:44.041618+0000 front 2024-03-22T19:20:44.041666+0000 (oldest deadline 2024-03-22T19:21:49.242225+0000) 2024-03-22T19:21:51.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:51 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:51.204+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-22T19:20:25.339106+0000 front 2024-03-22T19:20:25.339210+0000 (oldest deadline 2024-03-22T19:21:30.039106+0000) 2024-03-22T19:21:51.579 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:51 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:51.204+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-03-22T19:20:34.740434+0000 front 2024-03-22T19:20:34.740387+0000 (oldest deadline 2024-03-22T19:21:40.040296+0000) 2024-03-22T19:21:51.579 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:51 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11[78949]: 2024-03-22T19:21:51.204+0000 7f825f134700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-03-22T19:20:44.041618+0000 front 2024-03-22T19:20:44.041666+0000 (oldest deadline 2024-03-22T19:21:49.242225+0000) 2024-03-22T19:21:52.578 INFO:journalctl@ceph.osd.11.smithi167.stdout:Mar 22 19:21:52 smithi167 bash[90830]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-osd-11 2024-03-22T19:21:53.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@osd.11.service' 2024-03-22T19:21:53.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:53.398 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-22T19:21:53.398 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-22T19:21:53.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.a 2024-03-22T19:21:53.759 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:21:53 smithi086 systemd[1]: Stopping Ceph mds.a for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:54.259 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:21:53 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a[90448]: 2024-03-22T19:21:53.770+0000 7f9a1cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:54.259 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:21:53 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a[90448]: 2024-03-22T19:21:53.770+0000 7f9a1cc00700 -1 mds.a *** got signal Terminated *** 2024-03-22T19:21:57.759 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 22 19:21:57 smithi086 bash[118068]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-a 2024-03-22T19:21:57.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.a.service' 2024-03-22T19:21:57.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:21:57.925 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-22T19:21:57.925 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-22T19:21:57.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.d 2024-03-22T19:21:58.259 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:21:58 smithi086 systemd[1]: Stopping Ceph mds.d for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:21:58.759 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:21:58 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-d[89489]: 2024-03-22T19:21:58.294+0000 7f9282600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:21:58.759 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:21:58 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-d[89489]: 2024-03-22T19:21:58.294+0000 7f9282600700 -1 mds.d *** got signal Terminated *** 2024-03-22T19:22:00.259 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 22 19:21:59 smithi086 bash[118207]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-d 2024-03-22T19:22:00.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.d.service' 2024-03-22T19:22:00.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:00.458 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-22T19:22:00.458 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-22T19:22:00.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.g 2024-03-22T19:22:00.759 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:22:00 smithi086 systemd[1]: Stopping Ceph mds.g for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:01.259 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:22:00 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g[90061]: 2024-03-22T19:22:00.817+0000 7f9838a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:01.259 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:22:00 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g[90061]: 2024-03-22T19:22:00.817+0000 7f9838a00700 -1 mds.g *** got signal Terminated *** 2024-03-22T19:22:05.259 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 22 19:22:05 smithi086 bash[118344]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-g 2024-03-22T19:22:05.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.g.service' 2024-03-22T19:22:05.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:05.500 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-22T19:22:05.500 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-22T19:22:05.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.j 2024-03-22T19:22:05.759 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:22:05 smithi086 systemd[1]: Stopping Ceph mds.j for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:06.259 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:22:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-j[89100]: 2024-03-22T19:22:05.858+0000 7f3f65600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:06.259 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:22:05 smithi086 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-j[89100]: 2024-03-22T19:22:05.858+0000 7f3f65600700 -1 mds.j *** got signal Terminated *** 2024-03-22T19:22:09.759 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 22 19:22:09 smithi086 bash[118483]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-j 2024-03-22T19:22:09.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.j.service' 2024-03-22T19:22:09.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:09.926 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-22T19:22:09.926 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-22T19:22:09.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.b 2024-03-22T19:22:10.222 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:22:09 smithi131 systemd[1]: Stopping Ceph mds.b for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:10.222 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:22:10 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: 2024-03-22T19:22:10.184+0000 7f90fcb92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:10.222 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:22:10 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b[116605]: 2024-03-22T19:22:10.184+0000 7f90fcb92700 -1 mds.b *** got signal Terminated *** 2024-03-22T19:22:13.722 INFO:journalctl@ceph.mds.b.smithi131.stdout:Mar 22 19:22:13 smithi131 bash[122246]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-b 2024-03-22T19:22:13.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.b.service' 2024-03-22T19:22:13.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:13.768 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-22T19:22:13.768 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-22T19:22:13.768 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.e 2024-03-22T19:22:14.038 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:22:13 smithi131 systemd[1]: Stopping Ceph mds.e for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:14.472 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:22:14 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e[115919]: 2024-03-22T19:22:14.036+0000 7f416137c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:14.472 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:22:14 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e[115919]: 2024-03-22T19:22:14.036+0000 7f416137c700 -1 mds.e *** got signal Terminated *** 2024-03-22T19:22:17.643 INFO:journalctl@ceph.mds.e.smithi131.stdout:Mar 22 19:22:17 smithi131 bash[122386]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-e 2024-03-22T19:22:17.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.e.service' 2024-03-22T19:22:17.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:17.683 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-22T19:22:17.683 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-22T19:22:17.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.h 2024-03-22T19:22:17.970 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:22:17 smithi131 systemd[1]: Stopping Ceph mds.h for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:18.222 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:22:17 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h[116261]: 2024-03-22T19:22:17.968+0000 7fed9f82f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:18.222 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:22:17 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h[116261]: 2024-03-22T19:22:17.968+0000 7fed9f82f700 -1 mds.h *** got signal Terminated *** 2024-03-22T19:22:21.707 INFO:journalctl@ceph.mds.h.smithi131.stdout:Mar 22 19:22:21 smithi131 bash[122525]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-h 2024-03-22T19:22:21.713 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.h.service' 2024-03-22T19:22:21.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:21.747 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-22T19:22:21.747 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-22T19:22:21.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.k 2024-03-22T19:22:21.972 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:22:21 smithi131 systemd[1]: Stopping Ceph mds.k for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:22.472 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:22:22 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k[115576]: 2024-03-22T19:22:22.023+0000 7f84ec354700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:22.472 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:22:22 smithi131 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k[115576]: 2024-03-22T19:22:22.023+0000 7f84ec354700 -1 mds.k *** got signal Terminated *** 2024-03-22T19:22:25.683 INFO:journalctl@ceph.mds.k.smithi131.stdout:Mar 22 19:22:25 smithi131 bash[122665]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-k 2024-03-22T19:22:25.690 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.k.service' 2024-03-22T19:22:25.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:25.725 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-22T19:22:25.725 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-22T19:22:25.725 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.c 2024-03-22T19:22:26.079 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:22:25 smithi167 systemd[1]: Stopping Ceph mds.c for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:26.079 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:22:25 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c[85019]: 2024-03-22T19:22:25.963+0000 7f4e5ce38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:26.079 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:22:25 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c[85019]: 2024-03-22T19:22:25.963+0000 7f4e5ce38700 -1 mds.c *** got signal Terminated *** 2024-03-22T19:22:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Mar 22 19:22:29 smithi167 bash[91085]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-c 2024-03-22T19:22:29.604 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.c.service' 2024-03-22T19:22:29.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:29.641 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-22T19:22:29.641 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-22T19:22:29.641 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.f 2024-03-22T19:22:29.898 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:22:29 smithi167 systemd[1]: Stopping Ceph mds.f for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:30.328 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:22:29 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f[84676]: 2024-03-22T19:22:29.896+0000 7f982444f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:30.328 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:22:29 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f[84676]: 2024-03-22T19:22:29.896+0000 7f982444f700 -1 mds.f *** got signal Terminated *** 2024-03-22T19:22:33.578 INFO:journalctl@ceph.mds.f.smithi167.stdout:Mar 22 19:22:33 smithi167 bash[91225]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-f 2024-03-22T19:22:33.647 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.f.service' 2024-03-22T19:22:33.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:33.683 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-22T19:22:33.683 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-22T19:22:33.683 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.i 2024-03-22T19:22:33.938 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:22:33 smithi167 systemd[1]: Stopping Ceph mds.i for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:34.328 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:22:33 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i[85359]: 2024-03-22T19:22:33.936+0000 7f5bc457e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:34.328 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:22:33 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i[85359]: 2024-03-22T19:22:33.936+0000 7f5bc457e700 -1 mds.i *** got signal Terminated *** 2024-03-22T19:22:37.578 INFO:journalctl@ceph.mds.i.smithi167.stdout:Mar 22 19:22:37 smithi167 bash[91365]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-i 2024-03-22T19:22:37.636 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.i.service' 2024-03-22T19:22:37.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:37.671 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-22T19:22:37.671 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-22T19:22:37.671 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.l 2024-03-22T19:22:37.929 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:22:37 smithi167 systemd[1]: Stopping Ceph mds.l for b22c63ac-e87e-11ee-95cd-87774f69a715... 2024-03-22T19:22:38.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:22:37 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l[84336]: 2024-03-22T19:22:37.928+0000 7f9386671700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T19:22:38.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:22:37 smithi167 ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l[84336]: 2024-03-22T19:22:37.928+0000 7f9386671700 -1 mds.l *** got signal Terminated *** 2024-03-22T19:22:41.579 INFO:journalctl@ceph.mds.l.smithi167.stdout:Mar 22 19:22:41 smithi167 bash[91503]: ceph-b22c63ac-e87e-11ee-95cd-87774f69a715-mds-l 2024-03-22T19:22:41.628 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b22c63ac-e87e-11ee-95cd-87774f69a715@mds.l.service' 2024-03-22T19:22:41.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:22:41.664 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-22T19:22:41.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T19:22:41.922 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:22:44.813 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T19:22:45.005 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:22:46.467 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b22c63ac-e87e-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T19:22:46.652 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: b22c63ac-e87e-11ee-95cd-87774f69a715 2024-03-22T19:22:48.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:22:48.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:22:48.172 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T19:22:48.202 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-22T19:22:48.205 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767/remote/smithi086/crash 2024-03-22T19:22:48.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash -- . 2024-03-22T19:22:48.290 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T19:22:48.290 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-03-22T19:22:48.294 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767/remote/smithi131/crash 2024-03-22T19:22:48.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash -- . 2024-03-22T19:22:48.359 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T19:22:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-03-22T19:22:48.361 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767/remote/smithi167/crash 2024-03-22T19:22:48.362 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash -- . 2024-03-22T19:22:48.408 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T19:22:48.408 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-03-22T19:22:48.409 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-22T19:22:48.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-22T19:22:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:grep: /var/log/ceph/b22c63ac-e87e-11ee-95cd-87774f69a715/ceph.log: No such file or directory 2024-03-22T19:22:48.495 INFO:tasks.cephadm:Compressing logs... 2024-03-22T19:22:48.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T19:22:48.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T19:22:48.503 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T19:22:48.535 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-22T19:22:48.543 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-22T19:22:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-22T19:32:09.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T19:32:09.249 INFO:tasks.cephadm:Removing cluster... 2024-03-22T19:32:09.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:32:09.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:32:09.252 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:32:18.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:32:18.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:32:21.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:32:33.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:32:33.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:32:36.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:32:39.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:32:39.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:32:39.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:32:39.486 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2024-03-22T19:32:39.487 INFO:tasks.cephadm:Teardown complete 2024-03-22T19:32:39.487 ERROR:teuthology.run_tasks:Manager failed: cephadm Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 2024-03-22T19:32:39.489 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-22T19:32:39.503 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 2024-03-22T19:32:39.504 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-22T19:32:39.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:32:39.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:32:42.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:32:51.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:32:51.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:32:54.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:33:06.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:33:06.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:33:09.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:33:12.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:33:12.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:33:12.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:33:12.770 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2024-03-22T19:33:12.843 ERROR:teuthology.run_tasks:Manager failed: install Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/util.py", line 148, in ship_utilities yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 220, in install yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/util.py", line 150, in ship_utilities _remove_utilities(ctx, filenames) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/util.py", line 121, in _remove_utilities ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 2024-03-22T19:33:12.844 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-22T19:33:12.856 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-22T19:33:12.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:33:12.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:33:15.938 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:33:24.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:33:24.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:33:27.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:33:40.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:33:40.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:33:58.431 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:34:01.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:34:01.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:34:04.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:34:04.511 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2024-03-22T19:34:04.511 ERROR:teuthology.run_tasks:Manager failed: clock Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/clock.py", line 66, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/clock.py", line 72, in task cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 2024-03-22T19:34:04.512 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-22T19:34:04.524 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-22T19:34:04.525 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-22T19:34:04.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:34:04.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:34:05.803 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:34:14.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:34:14.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:34:14.950 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-22T19:34:19.046 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-22T19:34:26.165 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-22T19:34:36.261 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-22T19:34:49.397 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-22T19:35:05.698 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-22T19:35:59.263 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:36:11.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:36:11.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:21.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:36:24.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:36:24.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:27.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:36:27.422 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2024-03-22T19:36:27.423 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 165, in teardown self.archive_log() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 195, in archive_log self.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 2024-03-22T19:36:27.461 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-22T19:36:27.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-22T19:36:27.481 INFO:teuthology.task.internal:Duration was 2940.487878 seconds 2024-03-22T19:36:27.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-22T19:36:27.492 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-22T19:36:27.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:36:27.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:30.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T19:36:39.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:36:39.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:42.799 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-22T19:36:43.075 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T19:36:43.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T19:36:43.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-03-22T19:36:43.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T19:36:43.401 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-03-22T19:36:43.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-03-22T19:36:43.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T19:36:43.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:36:43.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T19:36:46.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-03-22T19:36:55.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:36:55.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:36:55.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-03-22T19:37:07.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:37:07.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:37:07.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-03-22T19:37:10.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:37:10.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:37:13.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-03-22T19:37:13.983 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2024-03-22T19:37:13.983 ERROR:teuthology.run_tasks:Manager failed: internal.syslog Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/syslog.py", line 76, in syslog yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/syslog.py", line 85, in syslog cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi167 2024-03-22T19:37:13.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-22T19:37:13.996 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-22T19:37:13.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T19:37:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:mv: cannot stat '/etc/sudoers.orig.teuthology': No such file or directory 2024-03-22T19:37:14.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T19:37:14.032 ERROR:teuthology.run_tasks:Manager failed: internal.sudo Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 452, in sudo yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 455, in sudo ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi086 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' 2024-03-22T19:37:14.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-22T19:37:14.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T19:37:14.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T19:37:14.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:37:14.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:37:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-03-22T19:37:14.118 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-03-22T19:37:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/home/ubuntu/cephtest/archive/coredump’: No such file or directory 2024-03-22T19:37:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2024-03-22T19:37:14.131 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/home/ubuntu/cephtest/archive/coredump’: No such file or directory 2024-03-22T19:37:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2024-03-22T19:37:17.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-03-22T19:37:26.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:37:26.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:38:26.059 DEBUG:teuthology.orchestra.remote:timed out 2024-03-22T19:38:38.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:38:38.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T19:38:38.151 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-03-22T19:38:38.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-03-22T19:38:38.268 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T19:38:38.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T19:38:38.270 ERROR:teuthology.run_tasks:Manager failed: internal.coredump Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 488, in coredump yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 491, in coredump run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi086 with status 1: "sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump" 2024-03-22T19:38:38.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-22T19:38:38.284 INFO:teuthology.task.internal:Transferring archived files... 2024-03-22T19:38:38.285 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi/7617767/remote/smithi086 2024-03-22T19:38:38.286 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T19:38:38.289 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-03-22T19:38:38.296 INFO:teuthology.orchestra.run.smithi167.stderr:find: '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2024-03-22T19:38:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2024-03-22T19:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory 2024-03-22T19:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-03-22T19:38:38.329 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/tarfile.py", line 2318, in next tarinfo = self.tarinfo.fromtarfile(self) File "/usr/lib/python3.8/tarfile.py", line 1105, in fromtarfile obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors) File "/usr/lib/python3.8/tarfile.py", line 1041, in frombuf raise EmptyHeaderError("empty header") tarfile.EmptyHeaderError: empty header During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 418, in archive misc.pull_directory(rem, archive_dir, path, maybe_compress) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 749, in pull_directory tar = tarfile.open(mode='r|', fileobj=r.stdout) File "/usr/lib/python3.8/tarfile.py", line 1633, in open t = cls(name, filemode, stream, **kwargs) File "/usr/lib/python3.8/tarfile.py", line 1514, in __init__ self.firstmember = self.next() File "/usr/lib/python3.8/tarfile.py", line 2333, in next raise ReadError("empty file") tarfile.ReadError: empty file 2024-03-22T19:38:38.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-22T19:38:38.347 INFO:teuthology.task.internal:Not uploading archives. 2024-03-22T19:38:38.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-22T19:38:38.360 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-22T19:38:38.360 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T19:38:38.372 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T19:38:38.375 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T19:38:38.381 INFO:teuthology.orchestra.run.smithi167.stderr:find: '/home/ubuntu/cephtest': No such file or directory 2024-03-22T19:38:38.383 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': No such file or directory 2024-03-22T19:38:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/home/ubuntu/cephtest’: No such file or directory 2024-03-22T19:38:38.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T19:38:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': No such file or directory 2024-03-22T19:38:38.391 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 138, in download_cephadm yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 346, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi086 was lost: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1753, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 1629, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 140, in download_cephadm _rm_cluster(ctx, cluster_name) File "/home/teuthworker/src/git.ceph.com_ceph-c_899aeb57f4ac58b35d5b52546466a4c7cb3bdc46/qa/tasks/cephadm.py", line 259, in _rm_cluster ctx.cluster.run(args=[ File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 522, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to smithi086 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi086 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-22T19:38:38.392 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-22T19:38:38.404 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-22T19:38:38.416 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: ConnectionError: Failed to reconnect to smithi086 2024-03-22T19:38:38.416 INFO:teuthology.nuke:Checking targets against current locks 2024-03-22T19:38:38.422 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/home/ubuntu/cephtest’: No such file or directory 2024-03-22T19:38:38.422 INFO:teuthology.orchestra.run.smithi131.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': No such file or directory 2024-03-22T19:38:38.441 INFO:teuthology.nuke:Not nuking smithi086.front.sepia.ceph.com because description doesn't match: rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi != /home/teuthworker/archive/yuriw-2024-03-22_13:10:42-rados-wip-yuri7-testing-2024-03-20-1625-quincy-distro-default-smithi/7616729 2024-03-22T19:38:38.466 INFO:teuthology.nuke:Not nuking smithi131.front.sepia.ceph.com because description doesn't match: rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi != /home/teuthworker/archive/yuriw-2024-03-22_13:10:42-rados-wip-yuri7-testing-2024-03-20-1625-quincy-distro-default-smithi/7616731 2024-03-22T19:38:38.487 INFO:teuthology.nuke:Not nuking smithi167.front.sepia.ceph.com because description doesn't match: rishabh-2024-03-22_17:46:08-fs:workload-rishabh-yuri-RunCephCmd-base-reef-testing-default-smithi != /home/teuthworker/archive/yuriw-2024-03-22_13:10:42-rados-wip-yuri7-testing-2024-03-20-1625-quincy-distro-default-smithi/7616727 2024-03-22T19:38:38.488 ERROR:teuthology.nuke:Could not nuke the following targets: targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1fLBDOwScRL51pal+CDrxMn8om72J7xA68fczNBRNIjdMYBx7xpHj740P2nJtvD3GaIbtx34EX7W4pE6BFqjg= smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6yoapWolvTnexwn4WKYy1CvVsJaJ2G5BKRISTNOlPNqyHDx3R/DuGJ8jL8daoCoDZ2aDqy+iWMk44SLmukBaI= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCRLZXAXZWAzIeSwaF0O7qnhuPPbcHyFWLJCNm1UF85AXrSd79OFiAUOFS+sggOdiqCTbaYO5tnr7xgcNihcyy4= 2024-03-22T19:38:38.491 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} duration: 2940.4878780841827 failure_reason: Failed to reconnect to smithi086 flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2024-03-22T19:38:38.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T19:38:38.580 INFO:teuthology.run:FAIL